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EXECUTIVE  SUMMARY 


Martin  Marietta  Energy  Systems,  Inc.  is  the  Management  and  Operating  Contractor  for  the 
Department  of  Energy’s  Oak  Ridge  National  Laboratory  and  other  Oak  Ridge  Federal  Facilities. 
The  Data  Systems  Research  and  Development  (DSRD)  Program  is  the  unit  of  Energy  Systems  with 
principal  responsibility  for  data  systems  work  performed  for  other  federal  agencies,  such  as  the 
Department  of  Defense.  DSRD  has  considerable  expertise  in  combat  modeling,  simulation  and 
gaming  and  in  performing  independent  verification  and  validation  of  combat  models.  Because  of  our 
expertise  and  our  independence  with  regard  to  the  Future  Theater  Level  Model  (FI  LM),  The  Joint 
Staffyj-8  asked  and  received  from  the  Department  of  Energy  our  aid  in  performing  an  independent 
verification  and  validation  study  of  the  FTLM. 

We  subjected  the  conceptual  design  of  the  FTLM  to  those  tests  that  we  thought  appropriate  to  its 
design  stage,  to  its  purpose  as  an  analytical  combat  model,  and  to  its  capabilities  as  specified  in  the 
Mission  Needs  Statement.  The  conceptual  design  passed  those  tests.  We  recommend  that  its 
development  be  continued. 

Because  this  recommendation  is  positive,  we  recommend  increased  attention  in  the  areas  of  design 
of  model  input  and  output  support  and  decision  logic  creation.  We  also  recommend  the  institution 
of  informal  configuration  management  control.  These  steps  are  appropriate  as  the  model  moves  to 
a  more  complex  and  costly  stage  of  development.  We  further  recommend  continuation  of  the 
planned  integration  of  independent  verification  and  validation  into  the  FTLM  design  and  construction 
process. 


Dean  S.  Hartley  III 
Principal  Investigator 
FTLM  IV&V  Team 
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ABSTRACT 


This  report  describes  the  methodology  and  results  of  independent  verification  and  validation 
performed  on  a  combat  model  in  its  design  stage.  The  combat  model  is  the  Future  Theater  Level 
Model  (FI  LM),  under  development  by  The  Joint  Staff/J-8.  J-8  has  undertaken  its  development  to 
provide  an  analysis  tool  that  addresses  the  uncertainties  of  combat  more  directly  than  previous  models 
and  yields  more  rapid  study  results. 

The  methodology  adopted  for  this  verification  and  validation  consisted  of  document  analyses. 
Included  were  detailed  examination  of  the  FI  LM  design  documents  (at  all  stages  of  development), 
the  FI  LM  Mission  Needs  Statement,  and  selected  documentation  for  other  theater  level  combat 
models.  These  documents  were  compared  to  assess  the  FILM  as  to  its  design  stage,  its  purpose  as 
an  analytical  combat  model,  and  its  capabilities  as  specified  in  the  Mission  Needs  Statement.  The 
conceptual  design  passed  those  tests.  The  recommendations  included  specific  modifications  as  well 
as  a  recommendation  for  continued  development. 

The  methodology  is  significant  because  independent  verification  and  validation  have  not  been 
previously  reported  as  being  performed  on  a  combat  model  in  its  design  stage.  The  results  are 
significant  because  The  Joint  Staff/J-8  will  be  using  the  recommendations  from  this  study  in 
determining  whether  to  proceed  with  development  of  the  model. 
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1.  INTRODUCTION 


This  report  describes  an  assessment  of  a  combat  model.  The  reason  for  the  assessment  was  to 
provide  independent  evidence  for  use  in  determining  the  future  development  course  of  the  model. 


1.1  BRIEF  PROBLEM  STATEMENT 

The  Joint  Staff/J-8  has  determined  that  a  need  exists  for  a  new  theater  level  model  of  combat  to  be 
used  as  an  analysis  tool.  That  model  is  currently  called  the  Future  Theater  Level  Model  (FILM). 
J-8  is  developing  the  model  using  standard  Department  of  Defense  techniques,  under  which  various 
milestone  criteria  must  be  met  before  proceeding  to  the  next  development  stage.  As  an  aid  to 
making  these  decisions,  the  procedures  for  this  model  require  independent  verification  and  validation 
of  the  model  during  its  development. 


1.2  PARTICIPANTS 

The  relevant  groups  involved  in  this  project  were  the  sponsor,  the  model  developer,  and  the 
performing  organization. 


1.2-1  FILM  Sponsor 

The  sponsor  of  this  project,  and  the  sponsor  of  the  FILM  development  project,  is  the  Force 

Structure,  Resource  and  Assessment  Directorate  (J-8)  of  The  Joint  Staff.  The  FILM  project  is 

located  within  the  Analytical  Tools  Program,  which  is  a  program  within  J-8. 

J-8  has  the  mission  and  authority  to: 

•  develop  and  improve  analytical  models,  techniques  and  procedures  used  in  studies  and 
analysis;  and 

•  develop  analytical  tools  for  use  by  the  unified  and  specified  commands. 

The  Mission  Needs  Statement  (MNS)  for  the  Analytical  Tools  Program  within  J-8  includes  the 

following: 

•  apply  operations  research  to  conventional  forces,  large  scale  ground  and  air  conflict,  theater 
level,  alternative  force  sizes,  structures  and  mixes;  and 

•  support  the  unified  and  specified  commands  capability  assessment  for  joint,  land,  sea,  and  air 
combat. 
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1.22  FI  LM  Developer 

The  sponsor,  J-8,  has  obtained  the  services  of  the  Operations  Research  Department  of  the  Naval 
Postgraduate  School  for  initial  conceptual  development  of  the  FI  LM.  The  Naval  Postgraduate 
School  has  the  intellectual  resources  of  its  staff  and  student  body  to  draw  from.  The  military, 
mathematical,  and  modeling  experience  that  these  resources  represent  was  judged  to  be  sufficient  for 
the  task. 


1.23  Performing  Organization 

Martin  Marietta  Energy  Systems,  Inc.  (Energy  Systems)  is  the  Management  and  Operating  Contractor 
for  the  Department  of  Energy’s  Oak  Ridge  National  Laboratory  and  other  Oak  Ridge  Federal 
Facilities.  The  Data  Systems  Research  and  Development  (DSRD)  Program  is  the  unit  of  Energy 
Systems  with  principal  responsibility  for  data  systems  work  performed  for  other  federal  agencies,  such 
as  the  Department  of  Defense.  DSRD  has  considerable  expertise  in  combat  modeling,  simulation 
and  gaming  and  in  performing  independent  verification  and  validation  of  combat  models.  Because 
of  our  expertise  and  our  independence  with  regard  to  the  Future  Theater  Level  Model  (FI  LM),  The 
Joint  Staff/J-8  asked  and  received  from  the  Department  of  Energy  the  aid  of  DSRD  in  performing 
an  independent  verification  and  validation  study  of  the  FI  LM. 


13  PURPOSE  OF  THIS  DOCUMENT 

This  document  constitutes  the  principal  deliverable  that  documents  our  findings  on  the  status  of  the 
conceptual  design  of  the  FILM.  It  contains  our  professional  opinion  on  the  verification  and 
validation  status  of  the  FI  LM. 


1.4  SCOPE  OF  THE  DOCUMENT 

This  document  describes  the  general  nature  of  the  project  and  the  standards  used  to  measure  the 
results.  It  describes  the  processes  we  used  to  test  the  FI  LM  against  the  standards  and  reports  the 
results  and  our  recommendations.  It  also  includes  the  detailed  comments  that  were  forwarded  to  the 
sponsor  and  to  the  model  designers. 


13  DOCUMENT  ORGANIZATION 

This  document  is  organized  into  six  major  sections.  The  first  section  (this  one)  is  an  introduction  to 
the  problem  and  the  participants.  The  second  section  provides  the  definitions  and  standards  used  in 
the  project.  The  third  section  describes  the  process  of  the  project.  The  fourth  section  gives  the 
overall  results  of  the  project.  The  fifth  section  contains  our  recommendations.  The  sixth  section,  an 
appendix,  contains  the  detailed  comments,  questions,  criticisms,  and  responses  that  were  developed 
in  the  course  of  the  investigation.  A  short  bibliography  of  verification  and  validation  literature  is 
included  in  the  end  matter  of  this  report. 


2.  DEFINITIONS  AND  STANDARDS 


To  our  knowledge,  no  previous  combat  model  has  undergone  a  formal  independent  verification  and 
validation  process  while  in  the  design  stage.  The  general  verification  and  validation  (V&V)  literature 
is  written  with  the  assumption  that  either  the  computer  model  exists  or,  at  least,  a  formal  design 
exists.  Neither  is  the  case  for  the  FTLM.  We  adapted  existing  procedures  and  definitions,  based  on 
our  experience  and  the  situation. 


2.1  DEFINITIONS 

Verification :  the  process  of  ensuring  that  a  (computer)  model  reproduces  the  processes  of  the 
conceptual  model  correctly. 

Validation :  the  process  of  ensuring  that  a  model  reproduces  reality  as  appropriate  to  its  intended  use. 

Independent  Verification  and  Validation  (IV&V):  a  verification  and  validation  project  performed  by 
a  party  other  than  the  designer  or  the  sponsor  of  a  model. 

IV&V  of  a  conceptual  design:  previously  undefined.  It  is  defined  in  the  next  two  paragraphs. 

According  to  the  definition  of  verification,  verification  of  a  conceptual  design  has  no  content. 
However,  for  convenience  in  separating  functions  and  by  analogy,  we  will  use  this  term  to  refer  to 
correspondence  of  the  conceptual  design  with  the  Mission  Needs  Statement.  The  operative  question 
is,  "does  this  design  satisfy  the  Mission  Needs  Statement?" 

Validation  of  a  conceptual  design  has  practical  problems  of  its  own.  Where  the  design  has  been 
sufficiently  refined  to  specify  algorithms,  we  can  determine  whether  the  validity  of  the  algorithm  is 
known  and  question  its  applicability  for  the  given  situation.  However,  where  the  design  is  less 
specific,  the  question  devolves  to  a  determination  of  the  likely  success  and  completeness  of  an 
approach  to  a  problem.  Other  issues  that  must  be  addressed  are  the  overall  completeness  of  the 
model  and  the  appropriateness  of  levels  of  detail  or  resolution  for  the  various  parts  of  the  model. 


22  VERIFICATION  STANDARDS  -  THE  MISSION  NEEDS  STATEMENT 

Die  MNS  for  the  Stochastic  Theater  Combat  Modeling  project  (which  is  the  initiator  of  the  FI  LM) 
defines  the  scope  and  vision  of  the  FI  LM.  The  FI  LM  Mission  Needs  Statement  is  oriented  on  the 
missions  and  authority  of  the  J-8  environment.  It  specifies  the  characteristics  that  the  FTLM  must 
have.  It  defines  the  attributes  of  the  FILM  by  listing  deficiencies  in  available  models  that  need 
addressing,  stating  current  capabilities  that  should  be  included,  enumerating  constraints,  and  defining 
milestones.  Thus  the  MNS  provides  the  standards  for  verification. 
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2.2.1  Modeling  DcGciencies  (Restated  as  FI  LM  Required  Features) 

The  FI  LM  should 

•  take  a  short  time  to  set  up,  run  and  analyze  many  and  different  friendly  and  enemy  courses 
of  action  (COAs),  operational  concepts,  threat  employment  scenarios,  and  force  mixes; 

•  incorporate  uncertainty  in  data,  scenarios  (multiple  regional  contingencies),  processes,  and 
represent  and  measure  variability  of  theater  outcomes; 

•  model  joint  and  combined  theater  of  operations  with  relatively  smaller  (compared  to 
European  scenarios  of  70’s  and  80’s)  and  highly  mobile  forces; 

•  incorporate  operational  command,  control,  communications  and  intelligence  (C3I)  explicitly, 
showing  effects  on  outcomes;  and 

•  easily  display  and  model  maneuver-based  warfare. 

2.2.2  Current  Modeling  Capabilities  (To  Be  Retained) 

The  FILM  should 

•  model  air  and  ground  combat  in  a  joint  theater,  with  naval  impact  on  ground,  littoral  and  air 
combat; 

•  model  deployment  and  sustainment  effects; 

•  model  effects  of  chemical  and  nuclear  weapons;  and 

•  model  effects  of  qualitative  factors,  such  as  leadership,  training  and  morale. 

2.2.3  Constraints  on  the  FI  LM 

The  FILM  must 

•  have  no  extensive  personnel  requirements  (J-8  personnel  can  set  up,  run  and  analyze  the 
model); 

•  use  J-8  standard  hardware,  software  and  operating  system  set,  but  also  be  portable  to  other 
hardware  and  operating  systems; 

•  be  written  in  Ada  -  unless  waiver  is  asked  for  and  given; 

•  have  a  complete  mathematical  description  and  tutorial  for  analysts  describing  how  to  convert 
the  analyst’s  war  to  the  model;  and 
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•  incorporate  independent  verification  and  validation  concurrent  with  design  and  test  of  the 
model. 


23  VALIDATION  STANDARDS 

Validation  standards  are  more  fluid  than  are  the  verification  standards.  Essentially,  the  standards 
consist  of  the  experience  and  knowledge  of  the  personnel  of  the  performing  organization.  This 
experience  and  knowledge  is  informed  by  the  current  state-of-the-art,  as  expressed  in  the  code  and 
algorithms  of  currently  accepted  combat  models.  However,  none  of  these  models  have  been  truly 
validated,  considerably  reducing  their  value  as  standards. 
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3.  PROCEDURES 


Because  no  procedures  for  doing  IV&V  of  a  conceptual  design  existed,  we  adapted  existing 
procedures,  based  on  our  experience  and  the  situation.  The  methodology  adopted  for  this  verification 
and  validation  consisted  of  document  analyses.  Included  were  detailed  examination  of  the  FTLM 
design  documents  (at  all  stages  of  development),  the  FTLM  Mission  Needs  Statement,  and  selected 
documentation  for  other  theater  level  combat  models.  These  documents  were  compared  to  assess 
the  FTLM  as  to  its  design  stage,  its  purpose  as  an  analytical  combat  model,  and  its  capabilities  as 
specified  in  the  Mission  Needs  Statement. 

We  created  a  ten  step  procedure  for  determining  the  contents  of  the  FTLM  design,  evaluating  the 
design,  and  reporting  our  findings.  These  steps  are  described  below.  As  part  of  standard  project 
procedure,  we  reported  each  month  on  progress  and  expenditures.  The  Gantt  chart  for  these 
procedures  is  shown  in  Fig.  1. 


FTLM  IV&V  Process  Chart 


Create  Team 
Study  Documentation 
Collate  and  Circulate 
Proactive  Opinions 
Obtain  Feedback 
Create  Draft  Report 
Revise  Draft  Report 
Deliver  Draft  Report 
Create  Final  Report 
Deliver  Final  Report 
Monthly  Reports 

Fig.  1.  Gantt  chart  of  FTLM  IV&V  Project 
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3.1  GATHER  A  GROUP  WITH  MANY  TALENTS 

The  idea  was  to  have  as  diverse  a  group  as  possible.  The  group  size  should  be  small  enough  to  be 
workable,  but  large  enough  to  ensure  diversity  of  outlook  and  coverage  of  the  technical  disciplines. 
We  ended  up  with  six  people.  The  skills  cover  the  areas  listed  in  Fig.  2. 


EXPERIENCE  WITH  COMBAT 
MODELS 

VECTOR 

CASTFOREM 

JTLS 

CBS 

AWSIM 

RESA 

NAVMOD 

TACWAR 

INBATIM 

Janus 

SIMNET 

SOTACA/ORGAME 


MILITARY  SKILLS 
intelligence 
C3 

special  operations 
general 

TECHNICAL  SKILLS 
math 

operations  research 
systems  engineering 
parallel  processing 
artificial  intelligence 
user  interfaces 
databases 
programming 


EXPERIENCE  WITH  ANALYSIS 
Army  models 
Air  Force  models 
Navy  models 
nuclear  effects  models 
joint  models 
capabilities  assessment 
historical  analysis 
systems  analysis 
data  analysis 

verification  and  validation 


Fig.  2.  IV&V  team  areas  of  experience. 


In  some  cases,  the  skills  groups  are  more  important  than  the  particular  skills  within  the  group.  For 
example,  the  primary  reason  for  needing  experience  with  combat  models  is  the  need  for  a  generic 
understanding  of  the  types  of  things  combat  models  do  and  a  knowledge  of  what  algorithms  are 
generally  accepted  by  users  for  various  parts  of  the  models.  Thus  knowledge  of  the  Army’s  CEM 
model  would  have  been  an  acceptable  substitute  for  knowledge  of  the  VECTOR  model.  The  military 
skills  group  is  also  important  in  the  same  fashion.  Experience  in  this  area  is  required  to  understand 
the  subject  being  modeled. 

In  other  cases,  the  particular  skills  within  the  groups  are  extremely  important.  Experience  with 
analysis  can  be  critical  in  determining  relevance  and  significance  of  design  issues  to  potential  model 
uses.  Some  technical  skills  will  fall  into  this  category,  depending  on  the  particular  model. 


32  STUDY  THE  FTLM  DOCUMENTATION 

Each  person  individually  read  the  central  materials  and  produced  comments  based  on  his  or  her  own 
conception  of  what  would  be  useful  to  say.  This  maximized  the  number  of  approaches  to  the 
problem.  Peripheral  materials,  such  as  documents  for  other  models  (TAC  THUNDER,  EAGLE, 
VIC,  CASTFOREM,  VECTOR,  ORSBM,  and  ORGAME),  and  historical  records  of  the  FTLM 
project,  were  distributed  so  that  each  was  read  by  at  least  two  members.  The  project  leader  read 
everything.  Each  comment  was  tied  to  a  model  concept  outline  to  permit  brevity  of  comment. 

A  list  of  the  documentation  is  included  in  the  bibliography  of  this  report. 
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33  COLLATE  AND  CIRCULATE  THE  FIRST  IMPRESSIONS 

We  collated  and  circulated  the  comments.  This  allowed  each  member  to  add  to  others’  comments. 
As  shown  in  Fig.  1,  this  step  overlapped  the  second  step.  Once  the  first  set  of  opinions  had  been 
entered,  the  value  of  independence  of  approach  was  confirmed  and  team  members  were  free  to 
modify  their  own  approaches  as  desired.  This  overlap  permitted  us  to  build  on  each  other’s  ideas 
rapidly  and  reduce  redundancy  to  some  extent. 


3.4  GENERATE  PROACTIVE  OPINIONS 

Members  were  asked  to  expand  their  view  from  reacting  to  the  documentation  to  examining  the 
broader  picture  of  what  should  be  in  the  concept  and  might  be  missing.  This  step  also  began  the 
process  of  looking  at  the  impact  of  the  projected  use  of  FILM  on  its  design  requirements.  This  step 
proceeded  concurrently  with  the  collation  and  circulation  step. 


33  OBTAIN  FEEDBACK 

Periodically,  the  work  in  process  was  discussed  with  FILM  developers  and  J-8.  This  permitted  some 
responses  to  questions  asked  by  the  team  prior  to  writing  the  report.  It  also  informed  the  sponsor 
of  the  status  of  the  work.  As  shown  in  Fig.  1,  this  step  started  after  the  first  part  of  the  collation  and 
circulation  step  and  extended  through  a  large  part  of  the  creation  of  the  final  report. 


3.6  CREATE  A  DRAFT  REPORT 

The  project  leader  created  a  unified  view  from  the  heretofore  disjointed  comments.  This  step  began 
at  the  end  of  steps  three  (collation  and  circulation)  and  four  (proactive  opinions).  Our  organization 
of  the  comments  (tying  them  to  a  model  concept  outline)  and  the  fact  that  they  were  already  in  a 
word  processing  document  made  this  a  relatively  short  process. 


3.7  REVISE  THE  DRAFT  REPORT 

The  draft  was  circulated  among  the  IV&V  team  for  comment  and  revised.  This  step  was  also  short. 
Tie  deadline  for  the  finished  draft  report  was  defined  by  the  need  for  J-8  to  use  it  in  the  Milestone 
1  review  of  FTLM. 


3.8  DELIVER  THE  DRAFT  REPORT 

The  final  draft  was  submitted  to  J-8  on  December  9,  1993,  in  time  for  its  Milestone  1  review. 
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3.9  CREATE  THE  FINAL  REPORT 

The  Milestone  1  Review  took  place  in  late  December  1993.  Using  feedback  from  the  Milestone  1 
Review,  we  have  revised  the  Draft  Report,  creating  this  report,  the  Final  Report.  This  process  was 
to  be  longer  than  that  required  by  the  step  to  create  the  Draft  Report,  as  shown  in  Fig.  1,  because 
it  included  format  editing,  formal  documentation  reviews,  and  clearance  and  release  procedures. 

Because  we  had  not  exhausted  the  funding,  J-8  asked  us  to  delay  delivery  of  the  final  report  until  the 
end  of  Fiscal  Year  1993.  The  reason  was  to  permit  any  significant  changes  to  be  reported.  No  major 
modifications  have  been  required,  although  some  stylistic  changes  have  been  made  to  the  document. 


3.10  DELIVER  THE  FINAL  REPORT 

The  final  report  is  being  submitted  and  distributed  during  August  1994. 


3.11  MONTHLY  PROGRESS  REPORTS 

Each  month  we  reported  on  the  activities  of  the  month,  the  planned  activities  for  the  next  month, 
and  the  expenditures  and  funds  remaining.  These  reports  supplemented  the  informal  discussions  and 
feedback  secessions. 


4.  FINDINGS 


We  put  the  results  of  the  independent  verification  and  validation  study  of  the  FILM  Conceptual 
Model  into  four  categories:  MNS  compatibility,  critical  flaws,  major  issues,  and  minor  issues.  The 
MNS  compatibility  issues  are  verification  issues  and  the  rest  are  validation  issues.  A  model  with  a 
critical  flaw  is  worthless  or  has  serious  restrictions  on  its  domain  of  value.  When  applied  to  a  model 
in  development,  this  category  means  that  immediate  corrective  action  is  required.  Major  issues  are 
issues  that  degrade  the  value  of  a  model.  The  implication  here  is  that  increased  attention  and 
resources  will  be  required  in  the  next  stages  of  model  development  to  prevent  such  degradation. 
Minor  issues  are  defined  as  areas  in  which  relatively  minor  efforts  are  required  to  correct  problems. 
Failure  to  correct  a  minor  problem  might  still  have  serious  effects  on  the  model’s  value.  The 
implication  for  the  FTLM  is  that  attention  needs  to  be  placed  in  an  area  to  prevent  the  actual 
occurrence  of  a  problem. 


4.1  MNS  COMPATIBILITY 

There  were  no  obvious  problems  with  satisfying  the  demands  of  the  MNS;  however,  not  all  MNS 
areas  were  explicitly  addressed  in  the  material  we  examined.  Our  verbatim  comments  are  given  in 
Section  6  of  the  Appendix.  Technically,  these  comments  represent  negative  verification  results; 
however,  these  are  not  serious  flaws,  given  the  state  of  the  design.  The  comments  are  summarized 
in  this  sub-section. 

Three  MNS  requirements  will  be  critical  determinants  of  the  ultimate  usefulness  of  the  FTLM.  They 
must  be  addressed  in  the  next  development  phase. 

•  One  FTLM  goal  is  to  reduce  the  total  study  turn-around  time.  There  should  be  metrics 
associated  with  this  goal.  Directly:  How  long  does  a  study  currently  take?  (A  range  of 
values  is  appropriate.)  How  long  should  an  FTLM  study  take?  Indirectly:  How  much  data 
is  required  for  a  current  study?  How  much  will  FTLM  require?  What  is  the  planned  FTLM 
run-time  for  one  replication?  How  many  replications  will  be  required?  How  many  different 
friendly  and  enemy  courses  of  action  will  a  typical  scenario  require?  The  use  of  a 
multiprocessing  computer  that  can  run  many  replications  simultaneously  may  be  part  of  the 
solution;  however,  attention  to  reducing  the  pre-  and  post-processing  times  is  mandatory. 

•  A  related  goal  is  a  requirement  for  a  small  support  team  to  run  FTLM.  Metrics  are  needed 
for  this  goal,  also. 

•  The  hardware,  software  and  operating  system  requirements  are  not  completely  clear.  Do  the 
requirements  for  "standard"  equipment  restrict  the  use  of  multiprocessor  computers  and 
enhancements  to  user  interface  equipment?  Do  the  Ada  requirements  prevent  the  use  of 
modern  object-oriented  programming,  which  appears  to  have  a  definite  place  in  the 
development  of  this  model?  We  recommend  considering  the  C++  language. 
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There  are  several  modeling  questions  that  have  been  left  open.  These  questions  are  less  challenging 
technically  than  those  addressed  in  the  conceptual  design  phase;  however,  they  need  to  be  answered 
in  the  next  development  phase. 

•  How  will  naval  impacts  be  implemented? 

•  How  will  variable  introductions  of  forces  and  materiel  and  variations  in  sustainment  be 
modeled? 

•  How  will  chemical  and  nuclear  weapons  be  modeled? 

•  How  will  qualitative  factors,  such  as  leadership,  training  and  morale  be  modeled? 

The  requirement  for  a  complete  mathematical  description  and  tutorial  has  not  yet  been  confronted, 
but  is  certainly  an  extremely  important  factor  in  making  the  final  model  usable. 

The  requirement  for  continued  IV&V  concurrently  with  design  and  test  will  be  very  important  in 
producing  a  practical  model. 


42  CRITICAL  FLAWS 

We  found  no  critical  flaws  in  the  design  of  the  FI  LM. 


43  MAJOR  ISSUES 

The  major  issues  can  be  grouped  into  four  categories,  Model  Design  Areas,  Decision  Logic,  Data 
Availability,  and  Management.  Our  verbatim  comments  on  major  issues  are  found  in  Section  7  of 
the  Appendix.  The  comments  are  summarized  here. 


43.1  Model  Design  Areas 

Four  major  design  areas  have  not  been  formally  addressed  in  the  FTLM  design  process.  We  can 
illustrate  the  problem  by  examining  the  current  structural  diagram  in  Fig.  3  and  comparing  it  to  a 
revised  structural  diagram,  shown  in  Fig.  4. 

As  shown  in  Fig.  3,  the  FTLM  design  documents  identify  the  areas  of  Analytic  Structure  and 
Environment  as  areas  with  comparable  importance  to  those  of  Maneuver,  Attrition,  C3I,  and 
Logistics;  however,  two  other  equally  important  areas  have  been  left  out:  Input  Tools  and  Data 
Storage.  In  Fig.  4,  the  four  operational  areas  have  been  grouped  together  and  the  four  areas 
concerned  with  input  and  output  (I/O)  have  been  grouped  together.  The  Analytic  Structure  area  has 
been  renamed  Analytic  Tools  and  refers  to  the  tools  used  for  analyzing  the  model  output.  The  Data 
Storage  area  label  indicates  that  a  database  management  system  (DBMS)  will  be  needed  for  data 
storage. 
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The  input  and  output  areas  have  been  addressed,  but  only  informally;  i.e.,  some  thought  has  been 
given  to  these  areas  and  concepts  have  been  put  forward  and  tested.  However,  the  level  of  attention 
has  been  relatively  low  and  must  be  raised  significantly. 


Fig.  3.  Current  architecture  diagram. 


Fig.  4.  New  architecture  diagram. 
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43.1.1  Input  tools 

One  of  the  major  things  to  be  learned  from  experiences  with  other  models  is  the  need  for  designing 
a  good  user  interface  at  the  beginning  of  model  development.  This  greatly  reduces  the  total  system 
cost,  decreases  the  number  of  people  needed  to  run  the  model,  increases  the  flexibility  of  the  model, 
vastly  decreases  the  turn  around  time  for  a  study,  permits  more  and  better  post-run  analyses,  and 
ensures  better  quality  by  reducing  errors  in  the  study. 

The  design  for  the  FTLM  user  interface  and  tool  set  needs  to  consider  the  types  and  volume  of  data, 
the  methodology  used  to  store  the  data,  the  hardware  and  software  constraints,  and,  most  importantly, 
what  the  FTLM  users  will  need  to  do.  Current  technology  indicates  the  solution  will  include  an 
object-oriented  graphical  user  interface  (GUI),  tied  to  a  flexible  DBMS,  with  access  to  graphical  and 
analytical  tools.  The  users  will  be  creating,  modifying  and  validating  the  FTLM  input  which  will 
include  geographically  referenced  data  and  multiply  connected  data.  One  tool  might  deal  with 
constructing  and  validating  the  networks;  another  might  aid  in  selecting  and  tailoring  units  from 
databases  and  placing  them  geographically  and  temporally;  and  a  third  might  check  for  consistencies 
of  various  sorts. 

Another  important  tool  that  will  be  needed  is  a  cataloger  of  input  datasets.  FTLM  may  require  a 
larger  set  of  scenario  variants  than  required  by  previous  analytic  models.  This  cataloger  would  aid 
in  the  creation  of  each  variation  (probably  as  a  shell  for  more  detailed  tools),  ask  for  verbal 
descriptions  (added  to  the  base  case  description),  and  create  names  for  each  variant.  Naturally,  it 
would  be  tied  to  the  analysis  tools. 

43.13  Data 

The  design  team  needs  to  begin  formulating  the  FTLM  data  management  plan.  The  FTLM  will  need 
storage  for  raw  data  to  be  used  in  creating  input  datasets,  storage  for  the  input  datasets  themselves, 
and  storage  for  output  datasets.  Considerations  of  importance  are  automatic  data  validation  support, 
support  for  update  through  graphical  objects  (for  instance,  changing  a  unit’s  geographical  position 
in  the  database  by  moving  its  icon  on  a  map),  ability  to  interface  with  the  input  and  output  functions 
of  a  model,  and  standard  DBMS  attributes  and  functions.  Object  Orientation  may  be  a  major  plus 
for  the  DBMS. 

43.13  Analytic  tools 

As  with  the  input  interface,  the  output  analysis  needs  to  be  planned  now.  The  measures  of 
effectiveness  (MOEs)  that  are  needed  to  interpret  results  need  to  be  defined  and  factored  into  the 
design  of  the  FTLM.  For  example,  if  territorial  ownership  over  time  is  such  an  MOE,  the  correct 
data  must  be  output  from  the  FTLM  to  support  the  MOE.  Ownership  can  be  calculated  using  the 
concept  of  neighbors  from  cellular  automata,  that  is,  node  (and  link)  ownership  depends  on  proximity 
to  and/or  time  since  last  occupancy  by  a  combatant.  Should  this  be  calculated  after  the  model  run 
from  a  history  of  the  nodes  or  should  it  be  calculated  within  the  model?  The  answer  partly  depends 
on  whether  node  ownership  information  would  be  useful  within  the  model,  e.g.,  in  making  certain 
decisions.  In  addition  to  specific  purposes,  the  FTLM  output  should  be  designed  with  interoperability 
with  a  statistical  software  package,  such  as  SAS. 
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43.1.4  The  environment 

This  model  design  area  has  been  only  loosely  defined  to  include  political  and  physical  constraints, 
such  as  excluded  territory,  airlift  capacity  and  weather.  Its  definition  needs  to  be  clarified  and 
integrated  with  the  other  areas.  There  are  probably  database  design  implications  and  modeling 
implications  that  need  attention. 


43.2  Decision  Logic 

A  major  issue  concerns  the  production  of  decision  logic.  This  will  be  an  extremely  significant  part 
of  the  working  of  the  entire  model.  Great  care  needs  to  be  exercised. 

There  is  not  enough  discussion  about  the  rule  sets,  how  they  are  specified,  implemented,  tested  and 
validated,  created,  modified,  and  maintained.  A  tool  for  decision  logic  may  be  needed  to  ensure  that 
each  hook  is  correctly  matched  when  a  rule  is  changed  and  that  no  extraneous  code  is  changed. 


433  Data  Availability 

This  is  a  major  issue  and  concerns  the  effects  of  data  availability  on  proper  design.  A  design  that 
does  not  take  data  availability  into  account  may  cause  serious  operational  problems  in  using  the 
model.  Missing  data  within  a  category  can  be  produced  by  interpolation,  e.g.,  a  data  value  for  a 
particular  weapon  type  can  be  derived  from  the  weapon’s  similarities  to  other  weapons.  However, 
a  missing  class  of  data  produces  a  conflict  in  combat  model  design.  Should  the  model  be  designed 
to  use  the  data  that  exist,  despite  flaws,  or  should  the  model  require  that  proper  data  be  created? 

The  volume  of  the  data  required  for  a  theater  level  model  necessitates  the  decision  that  only  small 
changes  can  be  made  to  the  existing  store  of  data  types.  Only  the  most  important  modeling  problems 
have  the  stature  to  justify  insistence  on  new  data  definitions.  All  other  data  requirements  must  be 
satisfied  by  existing  data.  Thus,  most  modeling  questions  must  be  driven  by  the  need  to  use  existing 
data. 

The  standard  must  be  that  data  inputs  must  derive  from  known  sources.  Exceptions  can  be  granted; 
but,  they  must  be  justified.  The  justification  should  be  the  importance  of  the  unique  data  and  the 
total  burden  (over  all  such  decisions)  should  be  extremely  small. 

A  task  needs  to  be  started  to  identify  available  and  required  data  types.  Sources  for  non-US,  non- 
NATO,  and  non- WARSAW  PACT  data  need  to  be  identified.  Both  current  and  "obsolete"  weapons 
data  will  be  needed. 


43.4  Management 

Two  issues  are  included  in  management.  These  issues  are  not  as  urgent  as  the  others;  however,  they 
do  need  to  be  addressed  in  time.  One  covers  documentation  planning.  The  other  covers  planning 
and  control  of  the  model  construction  process  itself. 
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43.4.1  Documentation 

The  MNS  identifies  a  need  for  documentation  of  mathematical  algorithms.  The  thrust  appears  to 
be  toward  ensuring  that  the  analyst  understands  how  to  convert  his  real-world  problem  into  a  proper 
FI  LM  problem.  This  is  the  correct  emphasis;  however,  it  needs  to  be  extended  beyond  mathematical 
algorithms  to  include  guides  for  other  areas,  such  as  creating  networks  that  model  desired  features 
and  creating  good  decision  rules.  It  also  needs  to  include  lessons  on  how  to  interpret  FI  LM  results. 

43.43  Control  of  model  development 

FILM  will  ultimately  need  a  formal  configuration  management  process.  Our  experience  has  been 
that  the  proper  time  for  instituting  this  is  when  there  is  a  completely  coded,  working  model.  The 
model  is  considered  to  be  working  in  the  sense  that  it  runs,  not  that  it  produces  the  correct  answers. 
Instituting  configuration  management  prior  to  this  stage  delays  progress  and  does  not  improve  results. 
However,  we  strongly  recommend  implementing  an  informal  configuration  management  process. 

The  informal  configuration  management  consists  of  a  set  of  lists  of  things  to  do.  The  issues  described 
in  this  section  and  detailed  in  the  Appendix  provide  a  starting  point.  The  concept  is  to  support  the 
proper  allocation  of  development  resources  by  the  technical  manager.  This  manager  can  use  the  list 
to  maintain  a  balanced  development  process.  Any  developer  with  a  software  process  maturity  level 
of  three  or  higher  will  realize  this  need  and  act  accordingly. 


4.4  MINOR  ISSUES 

Most  of  our  questions  have  been  discussed  with  either  the  sponsor  or  the  FI  LM  Project  Team. 
Those  that  we  determined  were  based  on  our  misunderstandings  have  been  omitted.  However,  some 
misconceptions  may  remain.  Some  of  our  questions  have  been  answered  satisfactorily,  but  are 
retained  to  confirm  those  answers  and  to  ensure  complete  communication  among  all  parties.  Some 
of  the  points  we  make  are  cautionary  in  nature,  rather  than  responses  to  parts  of  the  concept.  We 
have  attempted  to  organize  these  questions  and  comments  to  permit  easy  reference  to  the  part  of 
FTLM  that  is  impacted;  however,  some  areas  necessarily  interact,  making  perfect  alignment 
impossible.  Our  verbatim  comments  are  given  in  Sections  1-5  of  the  Appendix. 


43  TOTAL  QUALITY  MANAGEMENT 

Total  Quality  Management  (TQM)  is  a  far-reaching  concept.  In  industrial  production,  one  of  its 
corollaries  is  that  it  is  better  to  build  quality  into  the  product  than  to  try  to  inspect  it  in.  This  same 
concept  is  applicable  to  the  creation  of  a  combat  model.  The  standard  verification  and  validation 
techniques  are  designed  for  inspecting  quality  into  models.  That  is,  one  looks  for  flaws  in  a 
(completed)  model  and  tries  to  patch  it.  In  contrast,  the  design  and  construction  process  for  the 
FILM  includes  independent  verification  and  validation  throughout  the  entire  process.  This  is  an 
excellent  example  of  TQM.  Not  only  does  it  permit  the  early,  and  relatively  inexpensive,  option  of 
stopping  work  that  will  not  provide  cost-effective  benefits,  but  it  also  permits  the  modification  of  the 
product  toward  higher  quality  at  the  earliest  opportunity. 
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The  FTLM  Mission  Needs  Statement  is  another  example  of  TQM.  The  availability  of  such  a 
document  before  commencing  work  permits  review  of  the  justification  for  the  project.  Given 
approval  of  the  Mission  Needs  Statement,  this  document  is  the  basis  for  testing  the  model’s 
conceptual  design.  This  process  may  appear  to  be  only  common  sense;  however,  there  have  been 
many  large,  expensive  models  that  were  produced  without  a  formally  reviewed  Mission  Needs 
Statement,  or  even  a  clear  idea  of  what  the  model  should  do  and  why  it  should  do  it. 


4.6  CONCLUSIONS 

The  initial  goal  of  any  V&V  project  is  to  search  for  flaws.  It  is  not  surprising  that  a  report  on  the 
findings  of  such  a  project  is  full  of  model  problems.  These  findings  are  important  and  the  design 
team  of  the  FILM  needs  to  take  note  of  them;  however,  it  is  important  to  have  the  proper 
perspective.  In  general,  the  problems  that  we  have  found  are  not  errors;  they  define  objectives  for 
the  next  stage  of  the  development  process.  We  see  the  FILM  as  having  the  promise  to  merit  a  next 
stage  in  its  development.  Its  concept  is  a  good  concept,  one  that  addresses  real  needs  of  military 
analysis.  Further,  the  TQM  philosophy  being  used  in  its  design  increases  the  prospects  for  success. 
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5.  RECOMMENDATIONS 


Our  basic  recommendation  is  that  development  of  the  FI  LM  continue.  This  model,  in  concept  and 
in  initial  execution,  promises  to  deliver  new  and  much  needed  capabilities  to  the  analysis  community. 

Our  second  recommendation  concerns  the  major  issues.  Each  of  the  major  issues  involves  the 
allocation  of  resources,  both  monetary  and  management  attention.  These  issues  are  classified  as 
major  issues  because  of  the  leverage  they  have  on  the  quality  of  the  final  result.  Given  the  scarcity 
of  resources,  allocating  more  effort  to  these  areas  will  extend  the  time  to  completion  of  the  FTLM. 
However,  this  extension  is  justified  because,  without  early  and  meaningful  attention  to  these  issues, 
the  model  will  require  post-completion  modifications  that  will  be  time  consuming,  expensive  and 
unsatisfactory. 

Our  third  recommendation  concerns  the  minor  issues  and  the  congruence  of  the  FTLM  design  and 
the  MNS.  At  this  point  in  its  development,  the  FTLM  requires  an  informal  configuration 
management  process  (as  described  in  the  major  issues).  Each  of  the  minor  issues  and  each  missing 
element  of  the  MNS  becomes  an  element  of  a  list.  The  technical  manager  uses  the  lists  to  decide 
on  the  sequence  and  resolution  of  each  item.  The  discipline  will  pay  off  as  the  model  moves  into  the 
coding  phase.  Not  only  will  it  facilitate  transition  to  the  more  stringent  configuration  management 
required  for  coding  the  model,  but  it  will  also  make  the  procurement  and  management  of  the  coding 
significantly  easier. 

Our  final  recommendation  is  that  the  model  undergo  further  IV&V  as  it  develops. 
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APPENDIX:  VERBATIM  COMMENTS 


The  verbatim  comments  are  organized  into  seven  sections.  The  first  five  sections  are  directly  related 
to  a  taxonomy  of  the  FTLM  processes  derived  from  the  Future  Theater  Level  Model  (FTLM) 
Summary  of  Model  Concept.  These  sections  are  Introduction,  Architecture,  C3I,  Maneuver,  and 
Attrition.  The  sixth  section  deals  with  the  need  for  congruence  of  the  FTLM  design  and  the  MNS. 
The  last  section  covers  those  issues  that  we  determined  to  be  Major  Issues. 

The  portions  of  the  sections  in  normal  type  face  are  guides  to  the  process  or  issue  under  discussion. 
The  portions  in  italics  are  our  comments.  The  name  of  the  individual  making  each  comment  is 
shown,  flush  right,  at  the  end  of  each  comment.  Responses  from  the  design  team  are  shown  in 
SMALL  CAPS. 


1.  GENERAL  REMARKS 

1.0.1  My  biggest  problem  with  the  math  modeling  in  FTLM  is  with  the  justification  for 
the  various  equations  and  models.  The  report  on  ” Sensor  Fusion  Models  T  by 
D.  P.  Graver  and  P.  A.  Jacobs  does  give  a  reference. 

Also,  the  report  on  " STOCWAR  -  Some  Design  and  Implementation  Concepts " 
by  Sam  Parry,  15  June  1992,  does  mention  that  some  ideas  were  pulled  from 
TAC  Assessor  and  Combat  Evaluation  Model.  Other  then  these  few  references, 
very  little  is  said  in  any  of  the  reports  about  justification  of  the  models. 

In  reading  the  reports,  I  get  the  feeling "  that  what  they  are  doing  is  defining 
some  probability  functions  or  distributions  for  certain  variables  and  then 
applying  probability  theory  to  these  functions  to  combine  them.  But  how  are 
they  coming  up  with  the  probability  functions?  Seems  to  me  that  they  have 
" guessed "  at  some  functions  by  looking  at  what  mathematical  functions  behave 
in  the  way  they  think  the  variables  should  behave  and  then  adding  some 
" tweaking "  parameters  to  the  function  to  control  its  shape  more  closely.  For 
example,  on  page  7  of  the  report " Physical  Theater ",  15  Mar  92,  the  equation: 
h(r)  =  H  (1  +  (u*r)**pj  ' 

has  the  two  user  specified  " tweaking "  parameters  ut  and  pr  How  does  one 
specify  these  parameters?  Do  the  parameters  relate  to  any  real  work  parameters 
that  a  commander  can  relate  too.  Or  is  this  the  same  thing  I’ve  seen  in  other 
models  where  game  players  leam  to  think  in  terms  of  "what  value  do  I  give  these 
artificial  parameters  so  that  the  results  of  the  game  come  out  the  way  I  think 
they  should”.  To  me,  this  kind  of  defeats  the  purpose  of  the  simulation.  If  the 
results  of  the  FTLM  simulation  come  out  wrong,  you  just  go  back  and  tweak 
some  parameters  till  you  get  something  reasonable  when  is  actuality  something 
is  wrong  with  the  modeling  assumptions.  It  seams  to  me  they  have  a  parameter 
estimation  problem  but  no  real  data  for  estimating  the  parameters  or  for 
confirming  the  form  of  the  equations. 
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This  may  be  a  reasonable  approach  for  coming  up  with  a  theory,  but  is  it  the 
most  reasonable  way  of  coming  up  with  a  realistic  simulation.  Remember  the 
work  we  did  on  Simnet-T.  The  model  required  certain  parameters  formulated 
in  a  certain  way,  but  what  we  found  was  available  was  data  formulated  in  a 
very  different  way.  And  it  was  a  real  guessing  game  trying  to  derive  the  needed 
data  from  what  was  available.  My  concern  is  that  something  of  this  sort  will 
happen  in  the  FTLM  if  the  modeling  is  approached  from  a  purely  theoretical 
standpoint.  Has  anyone  taken  a  look  at  what  data  is  actually  available  and 
then  tried  to  build  a  mathematical  model  around  it??  Such  a  model  may  be 
less  sophisticated  but  may  have  a  real  basis. 

If  not  enough  data  is  available  for  doing  this,  then  maybe  another  approach 
would  be  feasible.  The  report  on  "Theater-Level  Combat  Modeling  ( Progress 
Report;  Feb.  1992)  by  Graver,  Jacobs,  Marsh-Jones,  and  Parry  advocates  using 
the  function: 

mik(t,  *)  =  (Bjdt)  /  R,(k;t)f  /  (1  +  (Bjfit)  /  Rfitqtjf  ) 
to  model  the  fraction  of  B’s  moving  to  node  k  by  t+1.  This  function  "says  that 
the  fraction  moved  is  an  increasing  function  of  B’s  perceived  force  ration  to  R". 
The  " tweaking "  parameter  p  controls  how  sharply  this  function  increases.  How 
is  p  determined?  How  about  using  a  pool  of  commanders  to  generate  data  for 
doing  the  parameter  estimation.  That  is,  give  each  commander  a  number  of 
scenarios  with  different  values  of  B,,  R]t  etc  and  ask  them  what  they  would  do. 
Then  use  this  data  to  generate  an  estimate  for  p.  If  you  can’t  get  a  good 
estimate,  then  maybe  the  form  of  the  equation  is  wrong  and  it  needs  to  be 
changed.  Then  when  this  is  used  in  the  FTLM  simulation  and  the  answers 
don’t  come  out  reasonable,  then  probably  something  is  wrong  with  how  the 
various  equations  were  combined  rather  than  with  the  parameter  value.  Kruse 
1.0.2  As  far  as  the  equations  they  have  used  go,  most  of  them  are  difficult  to  verify 
and  appear  to  be  guesses  at  the  functional  form  with  no  justifications  given.  I’d 
like  to  know  more  about  their  justifications.  I  did  see  a  few  equations  that  don ’t 
seem  right: 

"Scoring  by  Attrition  in  Combat  Models"  by  D.  P.  Gaver: 

1)  1  think  equations  3.4  and  3. 7  should  read  E[B(t+l ) \B(t)R(t) ] 
rather  than  E[B(t+l ) \R(t)B(t)j  to  mirror  equation  3.3 

2)  Equation  3.7  should  have  B(t+1)  rather  than  B(t-l). 

"Physical  Theater",  15  Mar  92 

1)  Page  6  uses  a  Uniform  distribution  for  estimates  of  the  Red 
force  by  Blue.  This  doesn’t  seem  very  realistic  to  me.  If  the 
Blue’s  intelligence  is  very  good,  they  should  have  a  tighter 
distribution  about  the  real  value  than  if  their  intelligence  is  not 
so  good.  Kruse 

1.0.3  ALGORITHMS,  HEURISTICS,  EQUATIONS,  AND  APHORISMS 

When  an  activity  is  well  defined  and  there  is  a  validated  formula  that  describes 
it,  that  formula  should  be  used.  Under  other  circumstances,  care  must  be  taken. 
Concatenations 

An  activity  may  be  modeled  with  an  approximation  based  on  little  knowledge; 
however,  such  an  approach  should  be  avoided.  For  instance,  assume  a  set  of 
serial  activities,  each  of  which  may  be  modeled  with  a  validated  formula,  except 
for  one  in  the  middle.  Using  a  guess  on  this  activity  places  the  model  of  the 
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composite  activity  at  this  same  low  level  of  fidelity,  despite  the  conditions  for 
each  of  the  other  activities.  It  would  be  better  to  replace  the  whole  composite 
with  a  single  guess  because  it  places  better  visibility  on  the  problem  and  may 
have  some  accessible  data  to  use  in  tuning  the  guess. 

Excessively  long  chains  of  reasoning  are  to  be  avoided  when  possible,  especially 
reasoning  that  requires  extensive  mathematical  skills.  Errors  are  less  likely  to  be 
caught  in  the  reasoning  because  there  are  fewer  people  who  can  follow  the 
mathematics.  Also,  long  chains  of  complex  reasoning  are  more  apt  to  be  coded 
wrong  and  the  error  remain  undiscovered. 

Complexity 

If  you  have  what  appears  to  be  a  very  clever  conceptual  model  and  it  is 
complex,  recast  it  with  an  approximation.  Check  the  two  against  each  other  and 
against  the  data.  If  at  all  possible,  use  the  approximation.  If  you  must  use  the 
complex  formulation,  explain  it  at  great  detail  with  copious  examples.  Assume 
your  audience  has  had  calculus,  but  only  remembers  it  vaguely.  If  you  can ’t  do 
that,  don’t  use  the  formulation. 

If  it  seems  desirable  to  decompose  an  activity  for  modeling,  it  is  better  to  first 
discover  the  data  available  for  testing  a  model  of  the  activity.  These  data  should 
drive  the  decomposition  into  modules,  as  well  as  the  depth  of  the  decomposition. 
It  is  better  to  have  a  demonstrable  approximation  to  a  process  than  to  have  a 
fine  theory  with  no  defendable  connection  to  reality.  This  is  true  for  two 
reasons:  the  error  level  of  the  approximation  is  more  nearly  computable  and 
data  to  drive  the  approximation  are  more  likely  to  be  available. 

Theories 

Theories  are  nice;  however,  remember  we  don’t  really  understand  war.  Your 
formulas  stand  as  propositions  to  be  proven,  not  by  mathematics,  but 
empirically.  If  there  are  no  data  to  fully  validate  the  proposition,  it  could  well 
be  false.  Remember  that  a  rifle  that  is  calibrated  and  precise  enough  to  have 
a  three  centimeter  shot  group  at  1000  meters  will  be  fired  by  a  human.  If  your 
proposition  is  true  given  a  set  of  circumstances  that  don ’t  include  a  pertinent 
battlefield  parameter,  it  is  not  only  not  useful,  it  may  be  dangerously  wrong. 
Data 

You  must  start  with  the  data  because  the  model  user  will  use  data.  If  your 
model  requires  data  that  don’t  exist,  how  will  the  user  cope?  The  user  must 
either  invent  the  data  or  not  use  the  model.  You  must  start  with  the  data 
because  that  is  all  you  have  to  test  your  model.  A  good  approximation  with  a 
known  error  level  is  better  than  an  finer  detail  model  with  no  information  on  its 
accuracy. 

Modeling 

What  do  you  do  when  there  are  no  data,  yet  you  must  create  a  model?  Start 
with  the  belief  that  your  model  will  be  wrong.  Your  goal  is  to  minimize  the 
significance  of  the  error.  A  simpler  model  will  be  easier  to  replace  if  a  better  one 
is  ever  found.  A  simpler  model  is  easier  to  describe.  A  simpler  model  will  have 
fewer  tuning  parameters  and  the  meaning  of  the  parameters  will  be  easier  to 
describe. 

Start  with  boundary  conditions.  You  may  be  able  to  get  strong  conditions  on 
your  model.  Next,  consider  all  possible  parameters.  Look  for  data,  including 
expert  opinion  on  the  nature  of  the  impact  and  crossed-term  effects  of  the 
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parameters.  Get  rid  of  any  parameter  that  appears  to  have  less  than  a  10% 
effect  on  the  model.  Look  for  reasonable  random  distributions  to  substitute  for 
missing  variables.  Write  an  explanation  that  shows  why  the  model  was  chosen 
and  shows  how  each  tuning  parameter  should  be  chosen,  what  the  bases  for  the 
decisions  are,  and  how  each  drives  the  model.  Be  very,  very  clear.  A  random 
result  that  is  replicated  many  times  is  preferable  to  a  deterministic  result  of 
unknown  value.  Hartley 

1.1.  PROBLEMS  WITH  CURRENT  THEATER-LEVEL  MODELS 

1.1. 0.1  Youngren  states  that  current  theater-level  models  fail  to  represent  the 
uncertainty  in  predicting  outcome.  That’s  probably  because,  at 
aggregated  levels,  no  one  knows  how  to  introduce  uncertainly  which  was 
aggregated  out  at  the  lower  levels.  Interesting.  Martellaro 

DISTRIBUTIONS  ARE  USED  TO  SUBSTITUTE  FOR  THE  LOSS  OF  DATA 
PRODUCED  BY  AGGREGATION 

1.1.1  Modeling  Imperatives 

1.1. 1.1  Many  input  values  are  unknown  and  unknowable 

1.1. 1.1.1  Not  all  probability  estimates,  especially  in  air-to-air 
combat  should  be  generated  by  the  model  and  then 
aggregated.  They  should  be  input  from  the  field. 
Experience  has  shown  that  values  calculated  from 
engineering  data,  using  standard  models  of  acquisition 
do  not  match  field  data  for  the  system  as  a  whole. 

Martellaro 

1.1. 1.1. 2  They  seem  to  be  basically  saying  that  they  see  the 

problem  and  they  know  it  exists  and  they  are  just  going 
to  treat  it  by  letting  everything  be  variable  instead  of 
fixed.  /  don ’t  see  any  discussion  about  where  they  will  be 
getting  the  values  they  do  use  for  various  parameters 
and  probabilities  they  are  using  in  algorithms.  And 
what  is  their  starting  point?.  Are  they  starting  with  a 
model  whose  philosophy  they  like  and  then  trying  to  find 
data  to  fit  it  or  are  they  looking  at  available  data  and 
trying  to  design  a  model  around  it?  Kruse 

It  IS  A  COMBINATION  OF  THE  TWO  POSSIBILITIES. 

1.1. 1.2  Operational  issues  have  more  effect  on  outcomes  than  tactical  issues 

1 . 1 . 1 .2. 1  I  believe  this,  but  is  it  faith  or  fact?  Hartley 

Reference  will  be  given. 


2.  ARCHITECTURE 

2.0.1  Figure  is  misleading  and  should  be  drawn  differently,  instead  of  showing  the 
Analytic  Structure  and  Environment  models  separately,  there  should  be  a  large 
box  drawn  around  the  CV,  Maneuver,  Attrition,  and  Logistics  models,  this  box 
would  then  represent  the  Analytic  Structure  and  Environment  models  since  all 
of  the  processes  occur  within  an  environment  and  must  be  developed  within  an 
overall  analytic  structure.  Turley 
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2.0.2  With  the  FTLM  there  is  an  opportunity  to  model  more  than  two  forces.  If  each 
side  is  represented  as  an  object,  there  should  be  no  theoretical  reason  that  n 
sides  cannot  be  represented.  (Practical  issues  such  as  execution  time  may  restrict 
the  size  of  n.)  This  permits  situations  where  there  is  an  active  population,  white 
force,  which  may  clog  the  transit  nodes  and  be  attrited  in  battles  and  air  strikes. 
It  also  permits  modeling  imperfect  alliances.  The  alliances  can  be  imperfect  in 
their  perception  of  allied  positions,  COAs,  etc.  and  be  imperfect  in  their  shared 
goals.  It  also  permits  resigning  from  the  conflict  by  a  side  (or  switching 
alliances).  In  a  multipolar  world,  this  capability  may  be  required.  Hartley 
2.0.3  Accounting  for  civilian  side  loyalties:  rules  could  be  created  that  represent 
support  for  a  combatant  side,  based  on  proximity  to  combatant  controlled  nodes, 
time  since  last  occupancy,  aggressiveness  of  recruiting  attrition  at  the  hands  of 
a  combatant,  etc.  Hartley 

2.0.4  Bottom  line:  develop  a  firm  concept  of  what  processes  are  properly  stochastic  at 
this  level.  I’d  like  to  see  a  chart  outlining  each  process  and  why  it  is  a 
candidate  for  variability.  When  that’s  done,  we  can  all  look  at  it  and  put  it  in 
perspective  with  respect  to  previous  models  versus  theater  level  planning 
concepts.  Martellaro 

2.0.5  Is  TAC  Thunder  being  used  in  its  present  form,  that  is,  are  they  doing  a  tie  into 
TAC  Thunder  and  doing  communication  between  the  two  models,  or  are  they 
going  to  modify  the  TAC  Thunder  source  code  to  fit  their  needs,  or  are  they  just 
going  to  use  TAC  Thunder  as  a  conceptual  starting  point  and  write  all  their  own 
code?  Kruse 

Only  some  parts  of  TAC  Thunder  logic  will  be  used.  There  are 

PROBLEMS  WITH  MANY  PARTS  OF  THAT  MODEL.  THE  LOGIC  WILL  BE  PART 
OF  FTLM. 

2.0.6  VIC’s  grid  cell  geometry  and  it’s  avenues  of  approach  methodology  for  ground 
maneuver  make  it  capable  of  modeling  surprise  and  military  objectives  other 
than  FEBA  movement.  Any  lessons?  Hartley 

2.0.7  Experience  with  CASTFOREM  has  shown  that  multiple  replications  can  be 
performed,  given  analyst  and  organizational  discipline.  Hartley 

2.0.8  Eagle  should  have  areas  of  commonality;  however,  its  execution  (as  described 
in  the  documentation )  is  so  poor  that  there  is  little  hope  of  learning  anything 
from  it.  Hartley 

2.0.9  JTLS  allows  multiple  sides.  There  is  no  reason  for  FTLM  to  fail  to  allow 
multiple  sides  also.  Hartley 

2.0.10  Several  mistakes  in  programming  structure  were  made  with  JTLS  because  of 
worries  about  the  wrong  computing  bottleneck.  Discussions  with  Rolands  & 
Assoc,  about  such  issues  might  help  prevent  similar  mistakes  in  FTLM. 

Hartley 


2.1  Terrain  Resolution 

2. 1.0.1  One  problem  with  the  design  of  the  FTLM  is  the  problem  of  indicating 
the  territory  that  is  held  by  each  side.  As  long  as  a  physical  node  or  a 
transit  node  is  occupied  by  a  unit  of  a  side  or  by  a  unit  from  each  side, 
the  ownership  is  clear.  In  a  totally  non-linear  campaign  this  may  be 
adequate  and  correct;  however,  in  a  conventional  or  semi-conventional 
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campaign,  some  territory  may  be  correctly  regarded  as  secure  even 
though  no  atom  sized  unit  occupies  a  node.  FTLM  could  use  some 
concepts  from  cellular  automata  to  rectify  this  situation.  Each  node 
(physical  and  transit)  could  have  an  ownership  attribute.  This  attribute 
is  set  by  proximity  to  units  and  to  other  nodes  and  by  history  by  some 
rule  set.  For  example,  if  a  node  is  occupied  by  units  of  only  one  side, 
it  is  owned  by  that  side.  If  occupied  by  units  of  multiple  sides,  it  is  in 
contention.  If  all  neighboring  nodes  are  owned  by  one  side  and  it  is 
unoccupied,  then  it  is  owned  by  that  side.  If  some  neighboring  nodes 
are  owned  by  one  side  and  others  by  another  side,  then  its  ownership 
attribute  decays  over  time  toward  the  side  owning  the  most  (or  most 
modified  by  closeness)  neighbors.  This  attribute  should  be  evaluated  in 
deciding  paths  taken  by  units.  Hartley 

2.1.1  The  ground  network 

2. 1 . 1 . 1  Need  to  include  algorithms  and  methods  of  defining  network  to  correctly 

model  everything  (like  in  Ground  Maneuver  paper)  Hartley 

2. 1.1.2  How  are  physical  node  capacities  handled  (multiple  arcs  meet  in  a  node 
of  lower  capacity =bottleneck  and  difficulties  should  ensue)?  Hartley 

2. 1.1. 3  User  input:  Document  says  ", Starting  node  with  planned  departure  time 

(this  will  be  the  initial  perceived  location)."  Does  this  imply  that  a  unit 
might  not  know  its  own  actual  location  or  just  that  its  HQ  might  not 
know  each  of  its  units’  locations?  Kruse 

2. 1.1. 4  How  is  planned  arrival  time  used?  Kruse 

2.1.2  The  air  network 

2. 1.2.1  Fig  3  shows  arc  crossings  that  are  not  nodes.  This  isn’t  consistent. 
Does  it  cause  problems  (aircraft  can  pass  with  no  interaction )? 

Hartley 

Air  interactions  will  be  tested  by  range  circles,  obviating 

THIS  PROBLEM. 

2.1.3  Connecting  the  air  and  ground  networks 

2.1.3.1  How  is  the  overlay  of  the  air  network  on  the  ground  network  used? 

What  if  the  arc  a  transit  node  is  on  is  partly  in  and  partly  out  of  the 
Reconnaissance  template?  Can  a  ground  node  or  transit  node  be  linked 
to  more  than  one  air  node?  Kruse 

2. 1.3.2  Be  careful  how  the  air  and  ground  network  grids  are  setup.  Make  the 

air  network  much  finer  and  capable  of  being  co-located  with  the  ground 
network  Otherwise  there  may  be  technical  problems  later  when  trying  to 
interface  ground  units  with  air  units  (Air  Defense  ->  aircraft)  or  Close 
Air  Support  (Air  ->  ground)  Martellaro 

2.1.4  The  logistics  network 

2.1.4.1  What  does  MSR  network  (the  logistics  network)  stand  for?  Hartley 

2. 1.4.2  You  say  the  MSR  network  is  a  subset  of  the  ground  maneuver  network. 
Why  not  a  super  set  with  extra  roads  too  small  for  brigade  maneuver  or 
an  intersecting  set  with  extra  roads  but  lacking  cross  country  arcs? 

Hartley 
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2.2  Unit  Resolution 

2.2.0. 1  Will  there  be  a  need  to  answer  questions  about  the  difference  that  a 
weapon  system  change  makes  (F-15  vs  F-22,  M1A1  vs  M1A2)?  If  so, 
this  will  have  an  impact  on  resolution  issues.  Hartley 

Resolution  will  generally  not  support  weapon  system 

LEVEL  QUESTIONS. 

2.2.0.2  In  the  Units  paper  (Draft  27  July  92)  you  discuss  attributes  and  in  the 
Strength  paragraph  mention  pointers  to  lists  of  attributes.  You  should 
consult  with  Ed  Kelleher  of  Rolands  concerning  pros  and  cons  of 
structures  for  unit  attributes.  He  has  mentioned  to  me  some  errors  he 
has  made  in  the  past  in  this  area  and  his  thoughts  since  then.  Hartley 


3.  THE  C3I  MODEL 

3.0. 1  PERCEPTION  AND  UNDERSTANDING  IN  FTLM 

On  the  real-world  battlefield,  perception  is  achieved  by  a  variety  of  means. 
Soldiers  see  (hear,  feel  or  smell)  things  directly  or  receive  data  from  mechanical 
sensors.  In  either  case,  what  they  sense  must  be  interpreted  before  being 
understood  (whether  correctly  or  incorrectly).  In  the  FTLM,  there  are  no 
individual  soldiers,  whether  privates  at  the  front  lines,  staff  intelligence  analysts, 
or  generals  in  the  theater  command  post.  There  are,  however,  three  possible 
levels  of  command,  exemplified  by  the  decisions  made  for  each  brigade, 
(potentially)  node-level  decisions,  and  theater  level  decisions. 

These  FTLM  decisions,  like  the  real-world  decisions,  should  be  informed  by  an 
understanding  of  the  situation,  not  raw  sensory  data.  Part  of  the  need  for  the 
FTLM  is  to  have  a  model  that  produces  a  spread  of  likely  results  of  combat 
scenarios  and  one  of  the  key  assumptions  is  that  command  decisions  are  an 
important  source  of  real-world  variability.  We  know  that  some  of  this  variability 
derives  from  differences  in  human  decision  making;  however,  we  also  know  that 
some  of  it  derives  from  the  fact  that  the  information  on  which  the  decisions  are 
based  is  inaccurate  and  subject  to  variation. 

Part  of  the  architecture  of  the  FTLM  is  based  on  this  variation  of  information 
about  the  state  of  the  (model)  world.  We  don’t  have  a  ready-made  statistical 
description  of  this  variation.  For  example,  we  can ’t  say  that  a  commander  from 
country  y,  on  the  defense,  in  hilly  terrain,  facing  a  brigade  from  country  x,  will 
perceive  his  enemy  as  a  company  with  5%  probability,  as  a  battalion  with  20% 
probability,  as  a  brigade  with  40%  probability,  as  two  brigades  with  30% 
probability,  and  as  a  division  with  5%  probability.  (In  addition,  the  description 
would  also  include  something  like  tank-heavy  or  not  and  likelihood  of  air 
support  and  amount  of  artillery,  splitting  the  categories.)  It  is  probable  that  no 
more  resolution  than  this  is  required  for  FTLM.  However,  in  an  effort  to  justify 
such  a  description,  more  detail  has  been  suggested. 

The  question  is,  " how  much  detail  is  required  and  is  it  supportable?"  The 
description  above  is  approximate  and  needs  to  be  confirmed;  however,  the 
concept  is  that  the  command  decision  is  based  (in  part)  on  an  understanding 
of  enemy  force  size  and  capabilities.  (The  example  above  suggests  another 
distribution,  that  is  the  commander’s  view  of  the  likelihood  of  other  sizes  for  the 
enemy,  given  his  base  estimate.)  Because  of  the  FTLM  architecture  of 
replicating  each  run,  different  methods  of  generating  the  description  above 
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should  be  regarded  as  equivalent  if  the  probabilities  are  within  10-15%  of  each 
other.  This  equivalence  should  affect  the  detail  required.  ( The  understanding 
of  the  enemy’s  intentions  [another  part  of  the  basis  of  the  command  decision] 
is  similarly  arrived  at  using  the  perception  of  positions  and  changes  in  position 
of  enemy  forces  over  time.) 

The  supportability  part  of  the  question  involves  data  availability,  data  quantities, 
and  model  run  time.  We  have  data  on  sensor  capabilities  at  the  engineering 
level.  Based  on  some  knowledge  of  the  TACSIM  model,  an  extremely  large, 
highly  classified  intelligence  model,  1  suspect  that  building  up  from  the 
engineering  level  of  data  to  the  level  required  for  the  FTLM  would  be 
unsupportable. 

The  approach  of  paragraph  three  either  averages  all  reasonable  actions  to  obtain 
information  or  includes  each  as  a  specific  part  of  the  case.  That  latter  situation 
would  be  very  clumsy  and  liable  to  error  in  enumeration.  It  appears  that  the 
FTLM  should  include  the  allocation  decisions  for  intelligence  gathering  of  some 
sort.  This  intelligence  gathering  will  have  a  variable  level  of  success.  However, 
the  interpretation  of  the  intelligence  will  also  have  variable  accuracy.  In  fact, 
an  incorrect  intelligence  report  can  be  incorrectly  assessed  to  give  the  correct 
answer.  I  don ’t  think  there  exists  an  algorithm  that  will " correctly "  fuse  sensor 
reports,  no  matter  how  well  they  are  modeled. 

I  would  ask  the  intelligence  community  if  it  could  and  would  answer  a  set  of 
questions  such  as  the  following: 

Given  a  set  of  sensor  packages  (or  appropriate  terminology),  given  a 
general  force  description  to  be  looked  for  ( the  enemy),  for  each  terrain 
type,  for  each  actual  occupancy  (empty,  civilians,  logistics  movement, 
squad,  company,  etc.), 

if  sensor  package  a  is  used  (normal  procedures),  what 
probabilities  would  you  associate  with  reporting  each  possible 
occupancy  state?  (This  is  a  semi-fused  result,  that  is,  you  would 
actually  combine  this  with  other  reports  to  produce  an  estimate; 
however,  assume  for  this  part  that  the  information  is  not  wildly 
at  variance  with  previous  conceptions.) 

If  the  answer  from  the  intelligence  community  is  "yes,”  then  I  would  decompose 
the  model  no  further.  The  Bayesian  methodology  for  combining  past  perceptions 
with  new  perceptions  is  mathematically  adequate,  understood  by  many,  and 
simple  enough  to  explain  to  the  rest.  If  the  answer  is  "no, "  then  I  would  ask 
them  to  describe  a  set  of  questions  they  could  and  would  answer.  Each  level 
deeper  requires  another  level  of  heuristic,  each  of  which  adds  complexity, 
possible  error,  and  sustainability  problems.  If  it  gets  too  deep,  I  would  build  a 
preprocessor  to  test  the  possibility  of  combining  states  that  are  equivalent  in 
result.  If  the  number  of  states  that  can  be  combined  is  large  enough,  a  table 
look  up  in  FTLM  will  be  more  efficient  and  the  separation  of  the  problems  will 
be  easier  to  explain. 

Finally,  the  proper  approach  to  perception  and  understanding  in  FTLM  depends 
on  what  data  are  available.  Hartley 

3.0.2  VIC  is  deterministic;  however,  it  was  specifically  designed  to  include  C3!.  Any 
lessons  learned?  Hartley 
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3.1 


Planning 

3. 1.0.1  It  would  be  worthwhile  to  examine  how  VIC  approaches  the  CO  A 
planning  problem  and  what  people  have  to  say  about  it.  Hartley 

3.1.0.2  CASTFOREM  also  demonstrates  the  need  for  intelligent  COA 
modification  and  testing.  If  (as  was  shown  in  CASTFOREM  and  Janus 
studies)  a  new  system  is  tested  without  accounting  for  differences  in 
tactics,  the  full  value  of  the  system  (or  its  problems)  may  be  missed. 

Hartley 


3.1.1  The  operational  concept 

3. 1.1.1  At  end  of  footnote  1 1 :  also  don’t  know  which  is  best  and  which  is  worst 
case  -  might  set  out  to  define  each  and  not  succeed.  There  might  be 
something  worse  than  worst  case  and  better  than  best  case.  Hartley 

3. 1.1.2  Does  the  " enemy  perception  about  our  operational  concept"  relate  to 
Western  values?  (As  an  extreme  example,  how  about  the  value  of  life?) 
Are  these  values  so  inbred  into  our  thinking  that  we  are  predictable  - 
despite  a  system  that  "seems"  to  offer  a  spectrum  of  COAs?  Relates  to 
the  technique  Grandmasters  used  to  defeat  the  first  generation  of  Chess 
programs  that  threatened  them.  About  1988-9,  mainframe  chess 
programs  started  playing  at  the  2200-2300  level.  Grandmasters  soon 
discovered  the  weaknesses  of  the  programs  and  purposely  led  the  chess 
program  into  these  areas.  Likewise,  in  any  " system ”  that  has  built-in 
constraints,  the  enemy  will  tend  to  discover  those  constraints  and  exploit 
them. 

Another  example:  A  15  year  old  Vietnamese  girl  is  no  match  for  a 
helicopter  full  of  Marines,  right?  -  until  she  smiles  sweetly,  approaches, 
and  then  tosses  a  hand  grenade  under  the  seat. 

Another  example:  the  Gulf  War  -  when  the  offshore  pump  was  opened 
and  sent  a  huge  oil  slick  towards  the  Saudi  de-salinization  plants. 
That’s  a  "crime"  that  we  would  never  consider,  but  which  the  Iraqi 
mentality  seemed  able  to  justify. 

Another  example:  Japanese  wargaming  predicted  that  after  the  attack  on 
Pearl  Harbor,  the  U.S.  would  surrender.  This  was  a  military  assessment, 
but  it  was  based  on  the  underlying  principles  of  their  culture.  Hence  it 
was  a  flawed  decision. 

I’d  like  to  see  a  discussion  of  what  assumptions  or  constraints  will  be 
made  -  both  for  us  and  for  an  opponent.  We  may  want  to  compare 
those  to  those  items  selected  for  variable  inputs  and  assess  their  relative 
importance.  All  courses  of  action  the  enemy  might  take  should  be 
surfaced  for  consideration,  even  if  we  would  not  consider  them 
ourselves.  Hence,  the  model  may  need  to  be  asymmetric.  Martellaro 

3.1.2  Defining  an  operational  course  of  action  within  FTLM 

3. 1.2.1  The  examples  in  each  of  the  sections  below  are  presented  as  instances, 

not  all  inclusive;  however,  we  might  want  to  add  some  to  make  sure  they 
aren’t  forgotten,  look  at  other  models  and  any  things  we  think  of 
ourselves.  Hartley 

3.1.3  Establishing  operational  concepts  under  uncertainty 

3. 1.3.1  Aren’t  timing  and  speed  of  progress  parts  of  COA?  Can  you  deceive 

about  either  or  both?  Hartley 
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3. 1.3.2  Are  deceptions  intentional  shifts  from  proper  COA  to  some  other 
possible  COA,  or  just  fogging  the  issue,  or  are  both  modeled?  Hartley 
Intentional  deceptions  by  shifts  must  be  part  of  human 
setup  of  COAs. 

3. 1.3.3  Document  says  "The  model  rule  sets,  as  well  as  actions  and  values 

scripted  by  the  analyst,  will  cause  the  forces  to  operate  in  various  ways 
according  to  the  relative  likelihoods  of  the  perceived  COAs.  Because  the 
perception  is  based  on  stochastic  processes  The  perception  is 
stochastic,  but  is  the  response  to  that  perception  stochastic  or 
deterministic.  That  is,  is  there  some  fuzziness  built  into  the  outcome  of 
applying  the  rule  set,  independent  of  the  fuzziness  of  input  parameters 
such  as  perception?  This  could  account  for  "intuition"  and  other  such 
intangibles  not  directly  modeled.  Kruse 

First  version  will  use  deterministic  responses. 

3. 1.3.4  Document  says,  "the  analyst  can  either  expand  the  rule  set,  insert  one  or 

more  external  events  to  force  particular  decisions  or  events  to  occur,  or 
stop  the  model,  make  manual  changes  to  the  data,  and  restart  it".  Does 
this  mean  that  expansions  to  the  rule  set  and  insertion  of  external  events 
is  made  on  the  fly  while  the  model  is  running?  Is  this  analyst  a  systems 
manager  type  or  just  a  player  type?  Is  there  one  for  each  side?  More 
than  one  per  side?  I  don ’t  get  a  good  feel  from  the  specification  what 
the  nature  of  any  "interaction"  with  the  model  is.  Kruse 

NO  CHANGES  DURING  REPLICATIONS.  ANALYST  SETS  UP  BOTH 
SIDES  FAIRLY  AND  MAKES  TEST  RUNS  TO  SEE  IF  ANYTHING  WAS 
FORGOTTEN.  CHANGES  ARE  MADE,  IF  NECESSARY,  INCLUDING 
FORCE  RULES. 

3.2  Reconnaissance 

3.2.0. 1  Classification  of  objects  as  military  may  overlook  key  entities  in  today’s 
(and  tomorrow’s )  scenarios,  e.g.  Somalia  -  the  object  and  objective  may 
be  a  refugee  camp,  food  distribution  station,  or  hospital. 
Counter-narcotics  Operations  -  objectives  may  be  crops,  processing 
laboratories,  or  civilian  transport.  Disaster  Relief  -  there  may  be  no 
military  (or  even  human )  enemy.  Packard 

FTLM  IS  NOT  AIMED  AT  THIS  LEVEL  NOW.  IT  IS  AIMED  AT  MRCS 
AND  MAY  OR  MAY  NOT  HAVE  APPLICATIONS  IN  THESE  AREAS. 

3.2.0.2  FTLM  needs  to  allow  multiple  sides.  This  permits  modeling  of  shaky 
coalitions  with  differing  objectives  and  civilian  populations  that  can 
suffer  attrition  and  might  shift  loyalties.  This  will  require  various  kinds 
of  objects  to  be  detected  by  recon.  Hartley 

3.2.1  Reconnaissance  Operating  Areas 

3.2. 1 .0. 1  As  I  understand  it,  each  reconnaissance  type  has  its  own 
template  type  and  their  can  be  numerous  templates  of 
each  type.  Do  the  various  templates  of  a  particular  type 
belong  to,  i.e.,  are  they  assigned  to,  a  particular 
reconnaissance  asset  or  to  the  reconnaissance  type? 
Can  two  reconnaissance  assets  of  the  same  type  share 
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a  template  either  at  the  same  time  or  at  different  times? 

Kruse 

3.2. 1.0.2  A  template  will  cover  or  not  cover  a  physical  node; 

however,  in  general  it  will  cover  only  part  of  a  transit 
node.  How  will  this  be  modeled?  Hartley 

This  is  under  discussion  and  not  decided. 

3.2.1. 1  Scheduling  times  of  detection 

3.2.1. 1.1  Document  says  "... modeled  with  periodic  detection 
opportunities  at  a  small  delta  t”.  Does  this  mean  that 
their  will  be  times  when  the  reconnaissance  assets  which 
provide  continuous  observation  will  not  be  able  to 
detect?  Kruse 

The  delta  t  refers  to  cyclical  update.  Since 

EVENT  PROCESSING  IS  USED,  THERE  IS  NO  IN 
BETWEEN  TIMES. 

3.3  Fusion 

3.3.0. 1  What  if  FTLM  fusion  is  better  (or  worse)  than  real  life  fusion 
process?  Hartley 

3.3.0.2  Fusing  recent  perceptions  and  then  updating  perception  database  may 
yield  different  results  from  incremental  fusion  of  each  perception  into 
database.  Both  processes  probably  occur  in  real  life  to  some  extent. 
How  is  this  to  be  handled?  Hartley 

Under  discussion. 

3.3.0.3  As  I  remember,  there  are  other  algorithms  for  combining  probabilities 
besides  Bayesian  and  derivatives  of  Bayesian  that  are  used  in  Expert 
Systems.  Have  any  of  these  been  considered?  Kruse 

Was  CONSIDERED;  BUT  PEOPLE  INVOLVED  ARE  BAYESIANS. 

3.3.0.4  VIC  models  explicit  sensor  data  collection  and  fusion  using  Kalman 
filtering.  The  data  are  used  to  support  production  of  current  target  lists 
and  a  perceived  situation.  Any  lessons?  Hartley 

3.4  Perception 

3.4.0.1  It  would  be  valuable  to  look  at  how  VIC  handles  perception,  looking  at 
pros  and  cons.  Hartley 

3.4.0.2  Perception  should  include  orientation  ( Where  is  the  combat  power 
focused?)  and  subordination  (Where  is  the  combat  support  element? 
Where  are  the  alternates?).  Packard 

3.4.0.3  The  need  for  logistics  presents  the  opportunity  to  explain  the  perception 
of  things  that  aren’t  there,  Le.,  tanks  at  nodes  where  there  are  really 
supply  trucks  or  civilian  trucks.  Hartley 

3.5  Decision 

3.5.0. 1  Is  the  model  run  interactively  at  all  or  is  the  only  interaction  when  the 
analyst  interrupts  the  game  to  modify  the  scenario  or  rule  set?  Kruse 
THE  LATTER  (HOWEVER,  THERE  MAY  BE  AN  INTERACTIVE  VERSION. 
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3.5.1  Decisions  to  be  made  by  the  operational  commander 

3.5.1. 1  How  about  decisions  to  deceive  enemy?  Hartley 

3.5.1.2  Decision  by  higher  commander  should  include  camouflage, 

concealment,  and  detection  (CCD)  measures  as  actions  that  a  unit  may 
need  to  participate  in  to  avoid  overhead  detection.  Turley 

Don’t  know,  is  a  resolution  issue. 

3.5.2  Missions 

3.5.2. 1  The  Assignments  for  Ground  Maneuver  Forces  assume  conventional, 
traditional  military  operations.  Humanitarian  relief,  emergency  reaction, 
counter-narc  are  overlooked.  NEED  A  REQUIREMENTS 
INTERPRETATION  HERE  -  THE  MNS  Para  3  DISCUSSION  Says, 

”  Changes  in  the  world  and  changes  in  operational  doctrine  require  a 
different  representation  of joint  force  employment,  explicitly  incorporating 
the  uncertainties  in  the  use  of  relatively  smaller  forces  in  a  less  densely 
populated  theater .”  How  important  are  the  humanitarian  relief, 
emergency  reaction,  counter-narc  missions?  Is  brigade  a  low  enough 
level  of  resolution  to  satisfy  the  MNS?  Packard 

Small  in  this  case  means  "not  8-10  corps,"  in  other  words  an 
MRC. 

3.5.2.2  Missions  should  include  such  forces  as  amphibious  operations,  special 

operations,  and  paratrooper  operations.  Turley 

Will  include  some  of  these  depending  on  resolution  (some 
forces  will  just  appear  where  needed). 

3.5.3  Assignments  for  ground  maneuver  forces 

3.5.3. 1  How  is  ”1.  Occupy  a  node  not  presently  controlled  by  own  forces” 

different  from  ”2.  Occupy  a  node  presently  controlled  by  own  forces”. 
The  two  descriptions  look  identical  to  me.  Kruse 

Typing  error,  needs  change. 

3.5.3.2  Is  it  a  reasonable  assumption  that  defenses  occur  only  at  physical  nodes 

and  delaying  actions  occur  only  at  transit  nodes?  Kruse 

3.5.3.3  FTLM  will  have  a  play  ahead  (internal  wargaming  the  situation)  feature 

in  the  ground  game.  This  is  analogous  to  the  kind  of  thing  done  in  the 
air  game  of  many  models,  that  is,  compute  the  best  path,  etc.,  based  on 
probable  outcomes.  This  has  been  done  in  air  games  because  it  is 
relatively  straight  forward  to  talk  about  single  probabilities  of  kill  (pKs) 
for  the  various  threats  to  an  aircraft.  It  is  more  complex  on  the  ground. 
Two  separate,  but  linked  issues  arise:  what  attrition  model  will  be  used 
by  FTLM  and  how  does  a  commander  estimate  the  situation.  Ignoring 
the  commander’s  estimate  for  a  minute,  the  look  ahead  attrition  model 
used  in  FTLM  should  be  related  to  the  actual  attrition  calculations  to 
be  performed.  In  this  way,  the  model  won ’t  pick  choices  that  are  dumb 
with  respect  to  how  it  is  going  to  evaluate  the  choices.  (The  basis  for 
the  choices,  i.e.,  incorrect  perception,  may  still  lead  to  poor  choices,  but 
that’s  life.)  On  the  other  hand,  if  real  commanders  use  (possibly 
incorrect)  rules  of  thumb,  regardless  of  the  actual  attrition  mechanisms 
in  war,  what  should  be  modeled?  Hartley 

3.5.3.4  In  planning  the  attack  and  choosing  possible  avenues  of  attack,  it  may 
be  that  the  commander  will  choose  the  one  with  the  greatest  opportunity 
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for  surprise.  (This  is  in  opposition  to  the  methodology  described  in 
Schmidt’s  thesis,  in  which  each  CO  A  is  examined  for  attrition  results, 
with  a  modifier  for  surprise.  Hartley 


3.6 


Communication,  Electronic  Warfare,  and  Deception 

3.6.1  Communication  in  FTLM 

3.6. 1.0.1  Can  you  say  the  impact  of  communications  lies  in  the 

impact  on  decisions?  Hartley 

3.6.1. 1  Degrade  the  ability  of  units  to  communicate  with  the  operational 
headquarters 

3.6.1. 1.1  You  say  it's  possible  to  summarize  detailed 

communications  models.  Is  that  really  so?  Who  is 
going  to  do  it?  How?  Hartley 

3.6. 1.1. 2  VIC  models  explicit  communications  networks  with 

loading  delay  and  priorities.  Any  lessons?  Hartley 

3.6.1.2  Degrade  the  ability  of  the  operational  headquarters  to  receive  and 
process  intelligence 

3.6. 1.2.1  You  say  that  operational  headquarters  is  assumed  to  be 

physically  in  the  rear.  In  a  non-linear  battle,  where  is 
the  rear?  Is  this  assumption  really  required  for  what 
you  say  next?  Hartley 

Physical  location  is  not  the  real  point  of  the 
discussion.  The  problem  is  that  a  single  point 

WILL  BE  ASSIGNED  FOR  THE  LOCATION  OF  THINGS 
THAT  ARE  REALLY  SPREAD  OUT  OVER  A  LARGE 

area.  This  means  that  a  single  bomb  would 

NOT  REALLY  DAMAGE  EVERYTHING.  ARTIFICIAL 
HARDNESS  MAY  BE  REQUIRED. 

3.6. 1.3  Degrade  the  ability  of  the  operational  headquarters  to  command  and 
control  forces 


3.6.1.3.1  The  figure  shown  on  p.26  (Summ,  Pt  l)is  not  labeled 

and  needs  to  be  for  clarification  (perhaps  it  is  the 
perception  database  updating  process).  Also,  the 
descriptions  for  transmission  delay  Til  and  T13  are 
identical.  Should  they  be?  Turley 

3.6.2  Modeling  communications  degradation 

3.6.2.0. 1  Communications  degradation  figure  shows  boxes  adding 
Tn  and  T»  to  delay  but  does  not  distinguish  them. 

Hartley 

3.6.2. 1  Affecting  the  quantity  of  communications 

3.6.2. 1.1  If  a  queue  gets  too  big  there  may  be  purposeful  or 

random  deletions  (or  selections  to  accept)  queue 
members.  Will  this  be  modeled?  Hartley 

TOO  SMALL  FOR  FTLM  RESOLUTION. 

3. 6.2.2  Affecting  the  timeliness  of  communications 

3.6.2.2.1  Communications  timeliness  should  reflect  the  changing 
scenario.  Lead  elements  into  a  theater  have  very 
limited  communications  unless  the  scenario  is  relatively 
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benign  to  allow  establishment  of  a  substantial 
communications  infrastructure.  Field  exercises  usually 
ignore  this  fact  by  sending  in  communications  (and 
other  support )  assets  as  much  as  a  month  ahead  of  the 
operators  and  conducting  a  COMMEX,  so  that 
communications  will  not  degrade  the  FTX. 
Communications  adequacy  is  changing  with  the 
introduction  of  each  new  or  modified  information 
system.  Communications  loads  vary  with  the  situation. 
” When  the  action  starts,  everybody  wants  to  talk  at 
once.”  Packard 

True. 

3.6.22.2  Document  says  "we  can  approximate  the  base  case  with 
a  time  td=0  and  focus  on  times  td  greater  than  the  time 
resolution  of  information  processing  to  reflect  the  effect 
of  communications  degradation"?  Where  does  the  value 
of  "time  resolution  of  information  processing"  come 
from.  Shouldn’t  the  base  case  be  for  td  =  time 
resolution  of  information  processing?  Kruse 

3.6.3  Modeling  Electronic  Warfare  (EW) 

3. 6.3.0. 1  How  localized  is  EW?  The  mesh  of  the  networks,  air 

and  ground,  will  impact  this.  The  localization  will  drive 
the  size  of  sigma.  How  much  of  the  info  that  is 
processed  is  impacted  by  EW?  I.e.,  to  what  does  the 
sigma  apply?  (Improper  sizing  of  meshes  can  make  it 
difficult  to  separate  what  is  included  or  include  what 
should  be  included  but  is  in  a  different  node).  Hartley 
Probably  will  not  do  EW  on  node  basis,  but 

ON  RANGE  BASIS. 

3.6.4  Deception  and  communications 

3.6.4. 1  Some  reference  to  use  of  commercial  news  media  for  both  intelligence 

and  deception  might  be  appropriate.  Packard 

3.6.4.2  Operational  Deception  didn’t  cover  some  of  the  more  common 

techniques  such  as  using  just-broken  codes  or  codes  suspected  to  have 
been  compromised,  false  news  releases  from  the  local  friendly  forces,  or 
supplying  true  but  misleading  information  to  news  organizations,  or 
appearing  to  take  as  face  value  and  act  on  enemy  propaganda  and 
misleading  information  (but  not  actually  doing  so.)  I  think  these  are 
important  at  the  theater  level.  Martellaro 


4.  MANEUVER 

4.1  Ground  Maneuver 

4.1.0.1  What  about  naval  and  riverine  operations?  Hartley 

No  NAVAL  VS  NAVAL  OPERATIONS  AND  RIVERINE  IS  BELOW  LEVEL 
OF  RESOLUTION. 
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4.1.1  The  ground  nodes 

4. 1.1.1  Logic  for  serial  traversal  would  allow  infiltration  and  pause  to  collect 

forces,  etc.  Hartley 

4.1. 1.2  What  about  islands?  Hartley 

4. 1.1.3  What  about  using  fuzzy  logic  for  node  suitability  for  enemy  missions? 

Hartley 

4. 1.1. 4  Assume  that  multiple  arcs  will  be  required  to  represent  transit  of  varied 

terrain,  varied  mode  of  travel  (mounted  vs  dismounted),  day  vs  night, 
forced  march  kt  cautious  advance,  single  route  vs  parallel  routes. 
Friendly  and  enemy  units  traversing  intersecting  transient  arcs  ( over 
which  the  average  travel  times  are  equal)  might  be  represented  as  having 
a  meeting  engagement  when  the  actual  travel  time  over  different 
segments  of  the  transient  arc  might  cause  them  to  not  encounter  one 
another.  Pages  2-3  resolves  this  adequately.  Packard 

4. 1.1.5  Are  node  characteristics  variable.  Can  engineers  alter  open  terrain  to 

rough?  Can  supplies  at  a  node  change?  Can  node  type  (mission  value) 
change?  Packard 

TO  BE  DETERMINED. 

4. 1.1. 6  Can  a  transit  node  become  a  physical  node  during  the  game,  either  by 

analyst  inten>ention  or  by  model  logic?  Kruse 

NO. 

4.1.2  Activities  that  can  occur  at  physical  nodes 

4. 1.2.1  Can  the  force  located  first  at  a  node  be  the  attacker?  Kruse 

YES. 

4.1.3  Activities  that  can  occur  at  transit  nodes 

4.1.3. 1  What  about  observation  and  attack  by  armed  reconnaissance  missions? 

Hartley 

NOT  CURRENT  US  MISSION  TYPE  -  HOW  ABOUT  ENEMY  OR  FUTURE 
CHANGES? 

4. 1.3.2  What  about  effect  of  obstacles  of  concentrating  a  delayed  force  in  a 

killing  zone  -  higher  attrition?  Hartley 

Think  about. 

4.1.3. 3  Might  want  prob<=l  for  meeting  engagements  to  include  cases  where 

arcs  are  really  ill-defined  because  terrain  allows  many  choices  (desert) 
and  forces  might  miss  each  other  or  might  not.  (This  is  a  reason  to  not 
simply  define  lots  of  alternate  arcs,  to  allow  a  force  to  detect  the  other 
and  move  to  attach  )  Hartley 

Not  sure.  Have  considered  very  dense  node  network  (like 
air  network)  for  these  situations. 

4. 1.3.4  How  about  meeting  engagements  by  forces  moving  in  same  direction,  but 

one  faster  than  other  (modeling  attack  from  rear)?  Hartley 

4. 1.3.5  The  assumption  that  a  meeting  engagement  will  occur  may  not  always 

be  valid.  Are  there  provisions  for  special  operations  forces  which  move 
along  the  same  transient  arc,  but  avoid  detection  (infiltration).  Are 
there  provisions  for  special  operations  forces  which  are  in  stay  behind 
mode?  Pg  8  para  b.  resolves.  Resolution  of  model  is  brigade  and 
higher.  Packard 
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4.1.4 


4. 1.3.6  Seems  to  contradict  pg  7,  para  4  by  permitting  passage  through  transient 
node  without  action,  however,  this  helps  resolve  my  concern  above. 

Packard 

4. 1.3. 7  Special  operations  forces  should  be  considered  here.  This  type  of  unit 

can  travel  undetected  along  the  same  transient  arc.  Turley 

Direct  insertion  of  forces. 

4. 1.3.8  Document  says  " Specifically ,  the  following  activities  can  occur  at 

physical  nodes:  1.  Deep  attack  strikes.  Any  physical  node ..."  Is  this  a 
typo  and  they  mean  transit  nodes?  Kruse 

Typo. 

4. 1.3.9  Can  a  meeting  engagement  occur  if  both  forces  are  moving  in  the  same 
direction  and  the  last  one  is  moving  faster  and  catches  up?  Kruse 

4. 1 .3. 1 0  If  I  understand  correctly,  a  C*I  asset  moving  along  an  arc  would 

be  a  ground  unit  with  C3!  capabilities.  Is  a  ”counter-C3I  attack " 
the  same  as  an  attack  on  any  ground  unit,  but  that  unit  just 
happens  to  have  C3I  capabilities,  or  is  there  something  special 
about  counter-Cfll  attacks?  Kruse 

EW  JAMMERS,  ETC. 

4.1.3.11  Document  says  " This  represents  reconnaissance  directed  at  a 

specific  area  along  the  arc  that  may  be  occupied  by  the  unit." 
An  arc  is  the  path  between  two  physical  nodes  represented  by 
one  transit  node.  Is  the  resolution  such  that  it  can  resolve 
specific  areas  along  the  arc?  How  is  this  done  when  the  arc  is 
represented  by  one  transit  node?  Kruse 

Concept  is  not  complete  yet.  They  are  considering 
SOME  KIND  OF  SUB-NODE. 

Maneuver  decisions 

4. 1.4.1  Movement  to  an  objective 

4. 1.4. 1.1  Can  the  user  change  paths  or  define  paths  during  the 

course  of  the  game?  When  does  a  route  get  dynamically 
replanned?  Kruse 

Path  -  yes;  network  -  no. 

4. 1.4.2  Splitting  the  force  among  two  or  more  paths 

4. 1.4.2. 1  What  about  splitting  force  for  serial  traversal  of  narrow 

path?  Hartley 

FORCES  MUST  BE  PREDEFINED  AS  TO  HOW  THEY 
SPLIT  (UNLIKE  SOME  OF  DOCUMENT  REFERENCES). 

Generally  brigade  will  be  lowest  level  of 

SPLIT. 


4.1. 4.2.2  How  about  notional  splitting  (produced  by  delay 

between  front  and  back  arrival )?  Hartley 

4.1. 4.2.3  When  splitting  forces,  to  what  unit  level?  For  parallel 

passage?  For  serial  passage  on  same  route?  For 
separate  missions?  Packard 

4.1.4.2.4  Camouflage,  concealment,  and  detection  (CCD) 
measures  should  be  included  as  actions  that  a  unit  may 
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4.1.5 


4.1.6 


need  to  participate  in  to  avoid  overhead  detection. 

Turley 


4. 1.4.3  Reactions  to  problems  encountered  enroute 

4. 1 .4.3. 1  In  delay  situations  may  want  to  include  different  arrival 

times  for  different  parts  of  a  unit  in  combat 
assessment?  Hartley 

Resolution  means  this  will  be  handled  in 

ADJUDICATION  OF  COMBAT  ONLY. 


4.1. 4.3.2  The  idea  of  penalizing  a  force  for  being  late  demands 
some  discussion.  One  could  argue  that  the  force  is 
discounted  in  combat  effectiveness  for  not  being  at  the 
objective  and  prepared  for  the  next  action,  which  seems 
to  be  the  author’s  rationale.  If  there  were  no  action 
planned  immediately  upon  arrival,  would  the  author 
then  cause  the  force’s  combat  effectiveness  to  return  to 
full  value?  Another  idea  is  that  arriving  on  time  or 
early  requires  speed  which  might  reduce  combat 
effectiveness  through  physical  exhaustion,  reduced 
control  or  vehicular  accidents.  Packard 


The  penalty  is  only  in  attrition  calculation 

AS  AN  EFFECT  OF  LATENESS,  IT  ISN’T  A  SEPARATE 
PENALTY  VARIABLE. 


4.1. 4.3.3  We  assume  the  transit  environmental  characteristics  for 

the  close  combat  model  would  include  degree  of 
darkness  and,  if  dark,  quality  of  and  competence  with 
night  vision  equipment.  Additionally,  terrain  should 
influence  the  percent  of  the  attacked  force  which  is 
exposed  and  can  participate  in  the  defense/counter 
attack.  Packard 

4.1. 4.3.4  Should  there  be  an  accompanying  EW  module  to 
calculate  the  final  strength  after  an  engagement?Turley 


Maneuver  variables 

4. 1.5.0. 1  Is  there  a  time  delay  for  other  kinds  of  attack  besides 

deep  attack?  Kruse 

4.1.5.1  Time  delay  due  to  encountering  enemy  action  (or  natural  obstacles) 

4.1. 5.1.1  Is  this  the  place  to  insert  nuclear,  biological  &  chemical 
(NBC)  effects?  Is  this  included  in  deep  attack? 

Packard 

4. 1.5. 1.2  What  is  the  justification  for  this  algorithm?  Kruse 
Terrain  effects  on  maneuver 

4. 1.6.1  If  an  arc’s  terrain  marginally  supports  passage  by  a  unit,  could  that 
same  terrain  be  less  trafficable  to  the  next  unit.  E.g.  a  heavy  force 
crosses  wet  farmland  or  grassland,  reducing  the  terrain  to  mud.  A 
dismounted  unit  (friendly  or  enemy)  attempts  passage  before  the  mud 
dries.  The  same  unit  attempts  passage  after  the  mud  dries.  Packard 

4. 1.6.2  What  about  NBC  warfare?  this  would  definitely  effect  ground 

maneuver.  Turley 
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4.1.7  Environmental  effects  on  maneuver 

4.1.7.1  What  about  effects  on  morale?  Hartley 

4.1. 7.2  Some  arcs  may  be  usable  (or  at  least  favor  use)  by  only  one  side 

( vehicle  type,  etc.)  Hartley 

4.1.8  CUTM-generated  arcs  for  seasonal  effects 

4. 1.8.1  Will  need  info  on  CUTM  to  detect  modeling  implications  of  CUTM 

choices  (e.g.,  what  does  it  do  for  desert,  where  you  can  define  paths 
going  anywhere)?  Hartley 

4. 1.8.2  What  is  CUTM?  Kruse 

CUTM  IS  AN  AUTOMATED  TERRAIN  ANALYSIS  TOOL  IT  WILL  BE 
REPLACED  BY  DEEM  (DYNAMIC  ENVIRONMENT  EFFECTS  MODEL) 
WHEN  DEEM  IS  FINISHED. 

4.1.9  Local  and  specific  weather  effects 

4. 1.9.1  Should  the  weather  effects  be  generated  from  historical,  meteorological 
files  or  taken  as  an  " average "  weather  pattern  for  that  season?  Turley 

4.1.10  Enemy  or  man-made  alterations  to  the  terrain 

4.1.10.1  Assume  that  Enemy  or  man-made  alterations  to  the  terrain  will 

include  obstacles,  minefields,  urban  rubble -ization.  Packard 


4.2  Air  Maneuver 

4.2.0. 1  This  section  is  well-written  and  well  thought  out.  Perhaps  the  ground 
maneuver  section  should  have  the  same  amount  of  detail,  Le., 
equipment  types  and  equipment  type  mission  assignments;  or  is  this 
already  taken  care  of  by  the  composition  of  the  ground  unit  assigned? 
Are  ground  units  "pre-packaged"  for  a  mission  or  is  it  necessary  to  pick 
and  choose  from  all  the  available  assets  ( like  the  admission  packages)? 
I  am  showing  my  Army  ignorance.  Turley 

4.2.0.2  The  concept  of  using  the  Dijkstra  algorithm  for  Air  units  using  minimum 
cost,  where  the  cost  is  related  to  the  Air  Defense  threat  is  exactly  correct. 
This  very  popular  now  in  flight  planning.  Martellaro 

4.2.0.3  How  is  airspeed  represented  when  nodes  are  gridded?  This  may  affect 
mesh  size.  Hartley 

Will  think  about. 

4.2.1  Mission  allocation 

4.2. 1.1  Missions 

4.2. 1 . 1 . 1  What  about  armed  recon  ?  Hartley 

4.2. 1.1. 2  Has  the  developer  addressed  the  possibility  of  flights 
being  reduced  during  or  just  prior  to  launch  or  are  all 
requested  flights  assumed  able  to  reach  the  rendezvous 
point  if  they  are  in  the  available  aircraft  list?  Packard 

4.2.1. 1.3  Document  says  "Rotary  wing  aircraft  MAY  be 
represented  as  a  strike  system  within  the  ground  model; 
in  this  role,  they  are  not  considered  in  the  air  model  of 
FTLM."  Can  rotary  wing  aircraft  play  other  roles,  and 
can  any  of  the  other  roles  be  part  of  the  air  model? 

Kruse 
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4.2.2 


4.2.3 


4.2. 1.1. 4 


4.2. 1.2  Aircraft  types 
4.2. 1.2.1 


Unlike  some  models,  CAS  will  be  in  the  air 

MODULE  AND  TREATED  AS  A  FORM  OF  STRIKE  TO 
PREVENT  DOUBLE  COUNTING  OF  AIR  POWER.  IN 
GENERAL,  HELICOPTERS  ARE  IN  DIRECT  SUPPORT  OF 
BRIGADES  AND  WILL  BE  TREATED  AS  ORGANIC 
PARTS  -  LIKE  ’FLYING  TANKS’.  HOWEVER, 
HELICOPTERS  CAN  BE  USED  AS  CORPS  OR  THEATER 
RESOURCE  AND  WILL  BE  PLAYABLE  AS  PART  OF  AIR 
OPERATIONS.  Care  will  be  needed  to  SUBTRACT 
THEM  FROM  BRIGADES  WHEN  THIS  HAPPENS  TO 
PREVENT  DOUBLE  COUNTING. 

If  a  certain  number  of  aircraft  are  used  for  logistics,  the 
air  model  can  simply  allocate  that  number  as 
unavailable.  If  they  are  subject  to  attrition,  a 
probability  distribution  can  kill  off  some  sometimes. 

Hartley 

What  about  AC130s?  may  be  needed  for  non-standard 
or  non-European  contingencies.  Is  this  a  problem? 

Hartley 

AC130S  CAN  BE  PLAYED.  THE  LIST  AS  GIVEN  WAS 
PRELIMINARY  AND  NOT  EXCLUSIONARY. 


4.2. 1.2.2  Is  this  the  right  way  to  handle  rotary  winged  aircraft?  It 
will  make  decisions  involving  them  difficult,  perhaps? 

Hartley 

4.2. 1.2.3  Are  particular  aircraft  types  to  be  played,  e.g.  F16s,  or 

will  there  be  notional  aircraft,  e.g.  " tactical  fighter?" 
Where  will  data  for  notional  aircraft  be  obtained  or  how 
will  it  be  derived?  How  much  data  will  be  required  if 
real  aircraft  are  played?  How  will  munitions  be 
modeled  (same  questions)?  Hartley 

4.2. 1.2.4  One  of  the  early  papers  suggests  notional  aircraft  such 

as  TF-CAS  and  TF  and  TF-INT,  how  about  Wild 
Weasels?  Hartley 

4.2. 1 .2.5  The  early  paper  suggests  the  possibility  of  having  CAS  as 

a  ground  forces  weapon  (I  presume  similar  to  JTLS). 
The  impact  of  such  a  decision  should  be  tested  by  trying 
both  explicit  CAS  and  implicit  CAS.  Some  problems 
were  observed  in  JTLS  (perhaps  because  CAS  could  be 
produced  both  ways.  Hartley 

Create  air  action  units  (sorties) 

4.2.2. 1  Mission  package 

4.2.2. 1.1  When  creating  an  actual  package,  will  there  be 

provisions  for  aborting  the  creation  because  of  lack  of 
numbers  or  critical  resources?  Hartley 

In  flight 

4.2.3. 1  How  about  effects  of  flight  profile,  nap  of  the  earth,  etc.  ?  Speed  over 

ground,  vulnerability,  detectability,  direction  of  attack?  Hartley 
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4.23.2  In  considering  flight  from  base  to  rendezvous  point,  consider  non-linear 

battlefield  where  enemy  can  be  anywhere  and  Israel  where  Israel  may 
control  air,  but  enemy  ground  forces  are  real  close.  Hartley 

4.23.3  Playing  entry  corridors  is  strongly  dependent  on  mesh  size  of  air 

nodes!  Hartley 

4.23.4  There  is  a  problem  with  defining  anti-air  defenses  only  at  nodes.  If  one 

side  has  a  doctrine  of  anti-air  integral  to  small  forces,  the  density  across 
the  theater  may  be  misrepresented,  allowing  too  free  access  to  air 
corridors.  This  is  especially  true  for  representing  more  conventional 
scenarios.  FTLM  should  not  be  restricted  to  answering  questions  only 
when  the  theater  looks  nothing  like  Europe.  The  question  arises  of  how 
best  to  represent  the  texture  of  the  air  defense  threat  in  a  3-dimensional 
view  (including  varying  vertical  ranges).  Hartley 

4.23.5  Too  small  a  mesh  might  mean  that  some  long  range  air-air  weapons 
lose  the  advantage  of  shooting  enemies  at  long  distances  (if  play  only 
shoot  enemies  in  own  node).  Other  considerations  say  use  smaller 
mesh,  may  have  to  consider  node  plus  ting  of  nodes  or  more.  Hartley 
AIR  AND  ANTI-AIR  WEAPONS  WILL  BE  PLAYED  WITH  RANGES  AND 
PERCENT  COVERAGE  OF  RANGE  CIRCLE  INTERSECTING  NODE 
SQUARE. 

4.23.6  Pg  8.  The  introduction,  ”We  begin  modeling  the  ingress  at  a  rendezvous 
point  over  friendly  airspace,”  may  not  always  reflect  a  valid  assumption. 

While  air  superiority  appears  to  be  a  given  in  most  scenarios,  there 
remains  the  possibility  of  mission  packages  being  attacked  before  or 
while  assembling.  Packard 

There  will  be  major  differences  between  FTLM  and  TAC 
THUNDER.  There  will  be  no  FLOT  and  so  no  guaranteed 

SAFE  TERRITORY. 

4.23.7  Pg  8.  ”This  generic  egress  model...”  in  the  discussion  of  Ingress, 

suggests  that  ingress  and  egress  are  treated  identically.  It  seems  that 
there  are  significant,  although  perhaps  equally  offsetting  differences 
between  ingress  and  egress.  During  egress  there  is  a  substantially 
reduced  element  of  surprise.  The  defenders  are  well  aware  of  aggressor 
aircraft  within  their  airspace.  During  ingress,  while  the  aggressors  enjoy 
greater  surprise,  the  defender’s  sensors  and  weapons  are  more  likely  to 
be  correctly  oriented.  Packard 

4.23.8  Chapter  2,  page  2-3,  paragraph  2.2.2,  Air  Mission  Sequence.  ”Flights 
may  originate  at  several  bases  and  rendezvous  at  specific  points.  The 
flight  group  then  crosses  the  FLOT,  where  combat  losses  may  occur.” 
The  concept  of  a  FLOT,  behind  which  forces  are  safe  is  not  necessarily 
valid  for  future  conflicts.  It  assumes  air  superiority  (probably  valid),  a 
friendly”  zone  from  which  aircraft  launch  and  rendezvous  (not 
necessarily  valid),  and  no  opposing  AA  capability  within  that  friendly 
zone  (wishful  thinking).  This  philosophy  does  not  allow  for  loss  of  air 
assets  due  to  enemy  action  at  the  forward  operating  base.  CBAM  or 
some  similar  model  might  rectify  this  apparent  shortfall.  Packard 
No  FLOT. 
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4.3 


4.2.3.9  Chapter  10,  page  10-7.  The  concept  of  a  FLOT  persists,  page  10-13. 
The  sanctity  of  the  FLOT  is  reduced  here  with  the  possibility  of  the 
FLOT  moving  or  of  the  base  being  damaged.  My  faith  is  restored. 

Packard 


Logistics  Movement 

4.3.0. 1  This  is  good  -  it  lends  itself  to  modeling  the  humanitarian  action. 

Packard 

4.3.0.2  The  Situation  #  4  -  Developer  should  be  sure  to  link  national  assets 
correctly  THINK  COMBINED  OPERATIONS.  Logistics  may  not  be 
interoperable  among  nations.  Packard 

4.3.0.3  Some  Suggested  Approaches  #i  -  not  all  logistics  originate  in  CONUS. 

Remember  combined  and  pre-positioned  assets.  Packard 

4.3.0.4  Some  Suggested  Approaches  #  2  -  there  are  some  extremes  in  port 
activity  which  developer  should  make  a  conscious  decision  to  model  or 
not  model.  One  extreme  is  CHAOS  -  nothing  can  be  identified,  wrong 
stuff  gets  forwarded  at  worst  time.  Another  extreme  is  perfection  -  the 
logisticians  can  rapidly  find  any  asset  and  load ,  dispatch,  and  transport 
exactly  what  is  needed  in  time  to  make  decisions  and  execute  operations 
with  confidence.  Packard 

4.3.0.5  Logistics  Feasibility  -  be  sure  to  get  a  real  logistician 's  opinion  to  off-set 
the  operators  tendency  to  shrug  off  the  complexities.  Things  just  don’t 
get  to  the  combat  units  in  nice,  evenly  distributed  packages.  Example:: 
if  half  the  convoy  is  food  and  half  is  ammo,  the  receiving  unit  is  not 
half  ready  when  all  the  food  arrives  -  he  is  totally  unready.  Packard 
4.3.0.6  Automated  Decisions  -Seems  like  this  would  be  very  helpful  in  initial 
force  laydown.  Rapid  scenario  setup  seems  to  be  a  major  goal  -  let  the 
analyst  place  an  element  of  a  unit  (say  the  forward  shooters)  and  have 
the  system  apply  a  notional  ( non-terrain )  spatial  distribution,  then  adjust 
for  terrain  and  adjacent  units.  Packard 


ATTRITION 

5.0.1  Attrition  may  involve  morale  and  experience  (cf.  training)  that  change  over 
campaign  Hartley 

5.0.2  Where  will  the  developer  refect  uncertainty  or  change  in  the  alignment  of forces 
encountered.  Armed  neutrals,  angry,  armed  civilians, or  hostiles  with  whom 
contact  is  forbidden  by  rules  of  engagement?  Packard 

5.0.3  "Attrition  modeling  is  overworked."  I  very  much  agree.  Martellaro 


5.1  Air  Attrition 

5. 1.0.1  When  air  units  are  returned  to  their  respective  bases,  how  are  the  losses 
going  to  be  divided  up  -  randomly,  proportionally,  or  other?  Kruse 
This  has  not  been  answered  yet.  Should  there  be  a  roll 

OF  THE  DICE  AT  EACH  AIR  NODE,  OR  CARRY  FORWARD  THE 
EXPECTED  SURVIVING  A/C  WITH  A  ROLL  TO  ROUND  AT  THE  END? 

There  will  be  an  adjudication  at  each  air  node  of  some 
SORT. 


48 


5.1.1 

5.1.2 


5. 1.0.2  Chapter  12,  page  12-3.  It  appears  that  attrition  of  friendly  air  is  based 
on  a  ” killed  or  unscathed”  philosophy.  Are  there  provisions  for  some 
aircraft  to  be  damaged  and 

1)  returned  to  base  for  future  use 

2)  continue  on  mission  with  degraded  capability 

3)  returned  to  base  and  aircraft  lost,  but  crew  saved 

This  is  not  particularly  important  in  assessing  the  day’s  combat  outcome, 
but  may  be  relevant  to  a  protracted  conflict  with  limited  air  assets. 

Packard 

Air-to-air  combat 

5. 1.1.1  Is  the  number  of  aircraft  lost  in  each  flight  a  stochastic  variable? 

Hartley 

Air  defense  modules 

5. 1.2.1  Too  large  a  mesh  means  that  too  many  ADA  can  engage  too  many 

aircraft,  too  small  a  mesh  means  that  too  few  ADA  can  engage  too  few 
aircraft;  however,  proper  ADA  vs  aircraft  mesh  may  not  be  proper  for 
air-air  engagements.  May  have  to  have  multi-node  search.  What  about 
edge  effects  ( aircraft  or  ADA  near  edge  of  area  and  can’t  be  shot  at  or 
shoot  at  near  but  unavailable  enemies )?  Is  this  important?  How  do 
you  tell?  Hartley 

WILL  USE  RANGE  CIRCLES  RATHER  THAN  NODE  PARING  FOR  ADA. 

5. 1.2.2  Short  range  air  defense  (SHORAD)  equipped  forces  by  units  too  small 

to  be  explicitly  represented  in  FTLM  can  be  represented  by  a  small  pK 
associated  with  each  node  and  link  based  on  proximity  to  combatant 
sides.  Hartley 


5.2  Close  Combat  Attrition 

5.2.0. 1  A  Cfl  question:  suppose  there  are  2  R  units  and  1  B  unit.  Situation  1 : 
the  Rs  have  good  C*/  links,  so  decisions  to  break  off  or  call  for  fires, 
etc.,  are  coordinated  single  decisions.  Situation  2:  the  Rs  have  bad  Cfl 
links,  so  decisions  are  separate,  based  on  each 's  estimate  of  situation. 
Are  both  situations  handled?  Hartley 

5.2.0.2  Heterogeneous  Lanchester  equations  are  very  difficult  to  assess  in 
practice.  Most  models  that  use  them  end  up  with  very  complex  attrition 
coefficients  that  involve  time,  distance,  allocation  of  fires,  posture,  etc. 
To  varying  degrees  the  factors  and  their  mathematical  insertion  are 
justified  verbally;  however,  it  is  not  clear  that  the  intended  effects  are 
accomplished. 

Suppose  the  problem  is  to  determine  whether  a  replacement  of 
tank  T1  by  80%  T1  and  20%  T2  is  a  good  idea.  One  approach 
would  be  to  run  a  model  that  allows  two  tank  types.  One  run 
would  have  100%  T1  and  no  T2;  a  second  run  would  have 
80%  T1  and  20%  T2.  However,  it  is  not  clear  that  every  model 
is  indifferent  to  splitting  forces  among  weapon  types.  One 
should  test  100%  T1  and  no  second  tank  types  against  80%  T1 
and  20%  T1  in  the  second  tank  position.  The  answers  should 
be  the  same.  Are  they? 
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Suppose  the  problem  is  sector  attack :  attacking  the  weak  sector 
of  a  defended  position,  rather  than  the  strong  sector.  The 
question  here  is  whether  in  setting  up  the  model  to  permit  sector 
splitting  actually  produces  the  effect  desired.  Suppose  you  are 
attacking  the  weak  side,  the  general  effect  should  be  to  cause 
higher  percent  casualties  to  the  part  of  the  defense  in  the  weak 
sector  than  would  be  the  case  on  that  part  of  the  total  defense 
if  it  were  part  of  an  undifferentiated  defense  or  part  of  the  strong 
sector.  Also  you  should  cause  unanswered  attrition  against  the 
rest  of  the  force  (for  a  while).  The  attacker  should  suffer  fewer 
casualties  than  if  attacking  an  undifferentiated  defense  or  the 
strong  sector.  The  complication  of  realistic  heterogeneous 
equations  may  make  this  dependent  on  settings.  (The  results  are 
not  obvious.)  Further,  in  the  real  world,  this  general  effect  may 
be  screw-up-able  (through  conscious  allocation-of-fire  type 
decisions  [not  just  through  screwing  up  the  attack]).  These 
points  need  to  be  tested  and  decisions  on  what  to  model  and 
how  to  model  it  taken  consciously.  Hartley 

5.2.0.3  But  one  danger  in  the  stochastic  output  -  ranging  from  the  " no  surprise " 
case  to  the  "great  surprise"  case  -  is  that  the  guidance  will  be  broad  and 
ill-defined.  I  worry  about  the  wisdom  of  letting  the  user  decide  for 
himself  whether  the  attack  will  be  a  surprise.  Is  this  a  useful  approach 
when  trying  to  introduce  variability?  Isn ’t  it  better  to  suppose  that  the 
enemy  will  not  be  taken  by  surprise,  and  if  successful,  simply  move  up 
the  timetable?  A  sensitivity  analysis  needs  to  be  done  to  determine 
whether  making  this  assumption  about  the  enemy’s  state  is  warranted. 
(This  is  a  separate  issue  from  the  reality  that  taking  the  enemy  by 
surprise  is  almost  always  good.)  More  exactly,  it  seems  that  the 
surprise/no-surprise  is  just  one  of  a  list  of  things  that  "seem  like  a  good 
idea"  to  introduce  without  any  sound  justification  for  it  being  a  variable 
factor.  (See  Architecture  about  making  lists.)  Martellaro 

USER  DOESN’T  DECIDE  IF  SURPRISE  IS  ACHIEVED. 

5.2.0.4  Surprise  can  be  a  major  factor  on  the  battlefield.  It  can  be  achieved  at 
the  tactical,  operational,  and  strategic  levels.  It  can  be  achieved  by 
either  side  (although  it  is  more  usually  achieved  by  the  attacker). 
FTLM  will  represent  certain  kinds  of  surprise  explicitly,  e.g.,  attack  up 
an  avenue  of  approach  that  the  enemy  considered  unlikely,  hence 
positioned  the  bulk  of  his  forces  elsewhere.  Presumably  (see  next  note) 
this  will  generate  its  reward  directly  by  greater  attrition  than  otherwise. 
However,  there  are  other  aspects  of  surprise  that  (apparently)  will  be 
represented  in  FTLM.  These  include  both  means  and  effects  (such  as 
greater  attrition  or  breaking  off  a  battle  early).  Some  ways  of 
representing  surprise  exist  (Schmidt’s  thesis,  Hartley’s  Oak  Ridge 
Spreadsheet  Battle  Model  [ORSBMJ);  however,  care  should  be  taken  to 
prevent  double  counting  the  effects  while  ensuring  that  these  other  effects 
are  represented.  Hartley 

5.2.0.5  In  the  heterogeneous  Lanchester  attrition  model  that  is  proposed  as  one 
of  the  alternatives  for  attrition,  the  complexity  will  require  detailed  testing 
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against  certain  flaws.  One  of  these  is  an  imperfect  implementation  of 
attacks  to  the  weak  side.  Unless  other  factors  dictate  otherwise,  an 
attack  to  the  weak  side  should  generate  greater  success  than  an  attack 
to  the  strong  side.  Representing  the  defender  as  occupying  subnodes 
which  are  engaged  differentially  is  one  approach.  However,  this 
approach  has  dangers.  For  instance,  suppose  the  attacker  has  three 
brigades  and  the  defender  has  four  brigades  at  the  node,  one  on  the 
weak  side,  two  on  the  strong  side,  and  one  in  reserve.  Note  that  direct 
calculation  of  a  strong  side  attack  shows  three  brigades  attacking  two 
brigades,  which  might  give  a  win  for  the  attacker  when  that  is  improper. 
This  could  be  a  result  of  mixed  resolution,  for  the  attacker  (perhaps) 
should  be  represented  as  sub-setted  with  two  brigades  up  and  one 
brigade  back.  Hartley 

5.2.0.6  FTLM  must  represent  the  propensity  of  a  defender  to  counterattack, 
given  the  opportunity.  Hartley 

5.2.1  Determine  forces  engaged 

5.2. 1.1  How  are  uncoordinated  groups  of  forces  and  partial  forces  against 

multiple  opponents  modeled?  Hartley 

5.2.1.2  If  a  new  force  arrives  at  a  node  in  the  middle  of  an  engagement,  do  you 

call  an  end  to  engagement  and  start  new  engagement  with  new  total 
forces?  In  some  cases  this  is  the  proper  answer.  Hartley 

5.2.13  How  are  indirect  fires  from  other  places  handled  in  battles  (considering 
shift  of  forces  within  a  node)?  Hartley 

5.2. 1.4  The  assumption  that  all  forces  in  a  transit  node  will  be  involved  is  bad 

news  for  modeling  ambushes.  Usually  the  ambusher  only  engages  a  part 
of  the  force  (part  of  success  story),  then  escapes.  This  is  part  of  reason 
to  use  small  capacity  (narrow)  but  long  arcs  that  extend  the  attackee. 
It  might  be  necessary  to  model  these  as  a  sequence  of  transit  nodes  with 
serial  splitting  of  force  passing  through.  Can  CUTM  handle  this  with 
automatic  chopping?  Hartley 

5.2.1.5  The  effect  on  morale  and  effectiveness  of  an  ambush  with  locally  heavy 

casualties  is  different  from  the  effect  of  the  same  number  of  casualties 
spread  throughout  a  brigade.  Hartley 

5.2.2  Determine  force  postures 

5.2.2. 1  In  determining  force  posture  for  computation  of  attrition,  the  developers 

should  consider  the  impact  of  a  force  operating  contrary  to  US  (or  any 
accepted)  doctrine.  For  example,  doctrine  defines  the  narrowest 
acceptable  movement  corridor.  A  force  might  choose  to  transit  a  much 
narrower  corridor,  either  for  expedience  or  due  to  lack  of 
experience/training.  This  might  result  in  a  force  posture  which  permits 
an  extremely  small  percentage  of  combat  power  applicable  to  an  enemy 
encountered  enroute.  Such  non-doctrinal  occurrences  seem  more  likely 
in  ” the  new  world.”  Packard 

5.2.3  Close  combat  attrition  object 

5.23.1  In  Footnote  8:  does  the  unit  "know"  its  own  numbers  (for  breakpoint 

calculations),  or  does  it  have  a  "perception”  of  these  that  may  be 
wrong?  Hartley 
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5.23.2  In  Footnote  10:  will  need  a  modification  of  ORSBM  for  expected 

arrival  of  reinforcements:  suppose  battle  time  is  going  to  be  2.5  days 
and  reinforcements  will  arrive  in  1.1  days.  Need  to  adjust  attrition  (and 
morale,  etc.)  for  1.1  days  of  fighting,  stop  the  engagement  and  compute 
new  one  for  new  forces  and  new  situation.  Hartley 

5.3  Strike  Attrition 

53.0.1  The  description  of  the  TAC  THUNDER  air-ground  attacks  in  one  of  the 
early  papers  appears  to  be  a  combination  of  calculated  probabilities  and 
stochastic  processing  (plus  some  expected  values  [cookie  cutter  weapon 
effects  radius]).  Combining  these  makes  the  algorithms  anything  but 
transparent.  Questions  of  efficiency  also  arise.  At  the  very  least,  a 
straightforward  walk-through  type  explanation  is  needed.  Hartley 

5.3.0.2  I  didn't  see  anything  I  disagreed  with.  One  of  the  variable  factors  not 
mentioned  is  fratricide.  (Mostly  air-to-ground.  Doesn’t  happen  much  in 
air-to-air.)  Martellaro 

53.1  Target  prioritization 

53.1.1  Strike  targets  are  prioritized  based  on  a  weighted  decision  function.  The 

example  formula  shows  the  prioritization  number  to  be  a  linear 
combination  of  perceived  probabilities  times  the  weight.  Can  other  types 
of  prioritization  algorithms  be  used?  Say  maybe  two  brigades  located  at 
a  node  is  worth  3  times  what  one  at  a  node  is  rather  then  just  2  times 
as  in  the  example.  Can  the  analyst  incorporate  such  changes  at  the 
beginning  of  the  game  and/or  during  game  play?  Kruse 

5.3.1 .2  The  same  comment  basically  holds  for  route  assignment  and  other  such 

processes.  In  other  words,  the  analyst  has  the  ability  to  select  between 
different  attrition  algorithms.  Will  such  selections  be  available  for  other 
algorithms  in  the  model?  Kruse 

5.3.2  Strike  mission  allocation 

53.2.1  Need  a  probability  of  success  (or  losses  incurred)  to  balance  value  of 
attacking.  If  you  are  shooting  artillery  at  a  target,  it  might  be  sufficient 
to  look  for  biggest  concentration  of  value;  however,  if  you  are  sending 
manned  aircraft  or  air  mobile  troops  at  target,  you  might  want  to  look 
for  big  enough  to  be  worthwhile  but  small  enough  to  win.  Hartley 

5.4  Logistics  Attrition 

5.4.0. 1  If  transport  is  subject  to  attrition,  a  probability  distribution  can  kill  off 
some  sometimes.  Similar  losses  can  be  applied  to  the  supplies  (both  air 
transported  and  ground  transported).  Hartley 

5.4.0.2  The  only  comment  I  have  is  that  accounting  for  the  interdiction  of  our 
own  supplies  is  critical.  The  interdiction  of  our  supplies  should  be 
viewed  as  something  the  opponent  will  focus  on  as  a  critical  failure 
point.  I  don't  have  a  problem  with  the  " backward-accounting "  approach 
mentioned,  so  long  as  it  is  integrated  into  the  expected  COAs  of  the 
opponent.  Martellaro 
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6.  MNS  ITEMS 

6.0.1  The  following  things  are  unknowns  (to  me): 

Time  required  for  a  study  (both  setup  and  execution). 

Current  set  of  J-8  tools. 

Components  or  elements  of  a  study  scenario. 

Resources,  methods,  and  tools  used  to  setup  and  run  a  scenario. 

Skills  and  capabilities  of  the  end-users. 

Implementation  of  the  corrections  to  the  identified  deficiencies. 

Computing  resources  identified  in  the  J-8  Analytical  Suite  Project  Management 
Plan  (PMP).  Thomas 

6.1  Short  Time  to  Set  Up,  Run  and  Analyze  many  and  different  friendly  and  enemy 
COAs,  operational  concepts,  threat  employment  scenarios,  and  force  mixes 

6.1.1  ambitious  project.  Specifically:  Rapidly  set  up  and  vary  concepts  and 

scenarios...  Packard 

6.1.2  How  much  time?  A  better  understanding,  via  example,  of  the  resources  required 

to  setup  and  run  a  job  would  be  nice.  Thomas 

Goal  is  run  time  of  about  30  minutes.  This  means  that  the 

REPLICATIONS  FOR  ONE  EXCURSION  CAN  BE  RUN  OVER  NIGHT  (USING  A 
MULTIPROCESSOR  WITH  ONE  REP  PER  PROCESSOR). 

6.1.3  How  many  is  many  different  possible  friendly  and  enemy  courses  of  action 

within  a  study  scenario?  How  different  is  different?  Thomas 

6.1.4  If  one  goal  is  to  reduce  the  turn  around  time  for  analysis,  there  should  be  a 

comparison  of  the  volume  of  input  (and  output)  data,  by  category,  of  current 
models  and  FTLM.  Hartley 

6.1.5  The  number  of  required  stochastic  runs  is  estimated  to  be  small.  That’s  a  pretty 
bold  statement  to  make  prior  to  an  analysis  of  the  typical  variancesMartellaro 

6.2  Incorporate  Uncertainty  in  data,  scenarios  (multiple  regional  contingencies), 
processes,  and  represent  and  measure  variability  of  theater  outcomes 

6.2.1  So,  is  there  any  way  to  design  parts  of  the  model  to  be  less  sensitive  to  input,  or 
are  they  just  going  to  handle  it  my  making  the  whole  model  itself  a  sensitivity 
analysis  tool.  Kruse 

Three  kinds  of  input  variables:  experimental  variables, 

EXPERIMENTAL  VARIABLES/UNKNOWN  FACTORS,  AND  REAL  WORLD 

VARIABILITY.  EXPERIMENTAL  VARIABLES  CATEGORY  CONTAINS  THOSE 
GIVEN  IN  THE  PROBLEM  STATEMENT  OR  DERIVED  FROM  THE  PROBLEM 
STATEMENT.  THE  QUESTION  IS  TO  DETERMINE  WHAT  DIFFERENCE  THEIR 
VARIATION  MAKES.  THE  EXPERIMENTAL  VARIABLES/UNKNOWN  FACTORS 
CATEGORY  CONTAINS  VARIABLES  THAT  NEED  TO  BE  VARIED  BECAUSE 
THEY  CANNOT  ALL  BE  CONTROLLED.  EXAMPLES  ARE  ACTUAL  ARRIVAL 
TIMES  (VS  PLANNED  ARRIVAL  TIMES)  OF  FORCES  IN  THE  FIRST  PART  OF 
THE  CATEGORY  AND  FIGHTING  EFFECTIVENESS  OF  FORCES  IN  THE  SECOND 
PART  OF  THE  CATEGORY.  REAL  WORLD  VARIABILITY  VARIABLES  INCLUDE 
WEATHER,  STOCHASTIC  ATTRITION,  AND  PERCEPTIONS  OF  REALITY. 
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6.3  Joint  and  Combined  Theater  of  Operations  with  Relatively  Smaller  (compared  to 

European  scenarios  of  70’s  and  80’s),  and  Highly  Mobile  Forces 

6.3.1  Clarify  " large-scale  and  complexity”  with  respect  to  force  sizes,  structures,  and 

mixes.  Clarify  ” small  highly  mobile  forces.”  Thomas 

6.4  Air  and  Ground  Combat  in  Joint  Theater  with  Naval  Impact  on  Ground,  Littoral  and 

Air  Combat 

6.4.1  How  will  naval  impacts  be  implemented?  Will  ships  be  nodes  in  the  sea?  Will 

they  move?  Hartley 

6.4.2  There  is  no  mention  of  a  naval  maneuver  model  (surface,  sub-surface, 

amphibious,  and  air  components).  Will  there  be  a  requirement  for  one?  1 
would  expect  the  amphibious  units  ( Marine  Corps )  would  become  ground 
maneuver  elements  when  they  landed.  J-8/CAD  has  NAVMOD  (at  least  they 
did  when  I  ran  it  back  in  1987).  This  model  may  have  been  incorporated  into 
TACWAR  by  now.  Turley 

6.4.3  Integrating  the  different  philosophies  and  workings  of  the  Air  Force  and  Army 

is  severely  glossed  over.  Should  a  new  metaphor  for  the  operation  of  FTLM,  in 
turn,  drive  the  military  thinking  about  cross-service  cooperation?  If  not,  then 
we’ll  end  up  letting  current  coordination  methods  remain  out  of  synch  with  a 
model  that  cannot  recognize  those  divisions.  Martellaro 

6.5  Incorporate  Operational  C3I  Explicitly,  Showing  Effects  on  Outcomes 

6.5.1  Ambitious  project.  Specifically:  Effects  of  C3 I  Packard 

Current  force  structure  questions  involve  substituting 

INFORMATION  TECHNOLOGIES  FOR  FORCE  STRUCTURE  SO  MUST  BE  ABLE 
TO  PORTRAY  COST  OF  POOR  INFORMATION.  THERE  WILL  BE  A  GREATER 

variation  of  scenarios.  And  need  distributions  of  output 

VARIABLES.  DRIVER  IS  C3I:  MEASURE  THE  CONTRIBUTION  OF  C3!  SYSTEMS 
BY  ONE  SIDE  OR  BOTH  AND  EFFECT  OF  LOSSES  TO  THESE  SYSTEMS;  SHOW 
THE  ROBUSTNESS  OF  A  FORCE  STRUCTURE  TO  DECISION  MAKING, 
OPERATIONAL  MANEUVER,  AND  OPERATIONAL  INTELLIGENCE;  SHOW  THE 
VARIABILITY  INHERENT  IN  THE  OUTCOME  WHEN  DIFFERENT  OPERATIONAL 
CONCEPTS  AND  PERCEPTIONS  ARE  REPRESENTED;  MAY  SUPPORT  BROAD 
MEASURES  OF  FORCE-LEVEL  EFFECTIVENESS  OF  C?I  CAPABILITIES.  WILL 
NOT  HAVE  RESOLUTION  TO  ADDRESS  MOST  WEAPON  SYSTEM  ISSUES,  E.G., 
Ml  A1  VS  Ml  A2;  HOWEVER,  MAY  ADDRESS  NEW  CAPABILITY  SYSTEMS,  E.G., 
LONG  RANGE  ABILITY  OF  MLRS  VS  NO  MLRS. 

6.6  Easily  Display  and  Model  Maneuver-Based  Warfare 

6.6.1  Ambitious  project.  Specifically:  combined  with  12.6  ground,  air,  &  naval, 

displaying  entities  which  function  at  radically  different  physical  speeds  and  with 
radically  different  effects  per  warrior  can  be  difficult...  just  to  decide  what  scale 
best  applies.  Packard 

Heart  of  question  is  MRCs.  E.g.,  consider  impact  of  2  MRCs  in 

TERMS  OF  LIFT  AVAILABILITY.  50-70  PHYSICAL  GROUND  NODES  PER 
THEATER. 

6.6.2  Clarify  the  requirements  for  ”easy”  (to  whom),  ”rapid”  (how  fast),  and 

" representation "  (what’s  best).  Thomas 
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6.7  Deployment  and  Sustainment  Effects 

6.7.1  How  will  variable  introductions  of  forces  and  materiel  be  computed  and 

handled?  Hartley 

6.8  Effects  of  Chemical  and  Nuclear  Weapons 

6.8.1  How  will  chemical  and  nuclear  weapons  be  modeled?  Hartley 

6.9  Effects  of  Qualitative  Factors,  such  as  Leadership,  Training  and  Morale 

6.9.1  How  are  these  factors  to  be  implemented?  Hartley 

6.10  No  Extensive  Personnel  Requirements  (J-8  can  set  up,  run  and  analyze  the  model 

themselves) 

6.10.1  References  to  end-users  are  inconsistent  (is  this  intentional  or  incidental).  Often 

referred  to  as  ” analyst",  " people ",  and " Joint  Staff  personnel  who  must  enter  data 
and  validate  results  without  support".  What  might  the  "without  support” 
mean?  Thomas 

6.11  Use  J-8  Standard  Hardware,  Software  and  Operating  System  Set,  But  also  Portable 

to  Other  Hardware  and  Operating  Systems 

6.11.1  "..  supportable  using  hardware,  software,  and  operating  systems  identified  in  J-8 

Analytical  Suite  PMP ..."  Can  we  get  a  description  of  that  hardware,  software, 
and  operating  systems  environment?  Packard 

6.11.2  Does  this  mean  no  proprietary  systems?  Does  this  mean  "only  commercial  off 
the  shelf"  products?  Does  this  mean  only  the  hardware  defined  in  the  PMP? 

Thomas 

Machine  restriction  is  posix  and  x- windows. 

6.11.3  All  the  output  calculations  must  be  output  to  the  user  in  a  very  special  way. 

There  should  be  excessive  and  healthy  discussion  about  how  to  best  display  this 
information.  Numbers  in  nine  point  type  on  a  40  cm  monitor  will  not  cut  it 
with  Generals.  Martel laro 

6.11.4  I  Urgently  and  Strongly  suggest  that  FTLM  get  away  from  small  monitors  and 

mice.  I  think  the  project  needs  to  think  about  other  display  and  input 
technologies  FOR  THE  END  USER.  It’s  OK  to  do  the  programming  with  CRTs 
and  mice.  It’s  not  OK  to  force  that  technology  onto  Generals  and  Colonels. 
Otherwise,  they’ll  just  finance  a  new  system.  As  I  understand  it,  instead  of 
techies  spending  months  setting  up  scenarios,  this  tool  is  to  be  used  by  high-level 
leadership  in  an  interactive  and  stochastic  mode.  They’ll  need  a  different  kind 
of  system.  I  recommend  something  like  large  horizontal  LCD  maps  with  "touch 
and  drag"  manipulation.  Martellaro 

6.12  Must  Be  Written  in  Ada  -  Unless  Waiver  Is  Asked  for  and  Given 

6.12.1  Does  Ada  operate  in  that  environment?  Packard 

6. 12.2  Ada  does  not  support  Object  Oriented  Programming  fully.  What  impact  will  this 

have?  Hartley 

DMSO  IS  HINTING  THAT  ADA  REQUIREMENT  WILL  GO  AWAY. 

6.12.3  I  strongly  recommend  C++.  Martellaro 
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6.13  Must  Have  Complete  Mathematical  Description  and  Tutorial  for  Analysts  in  How  To 

Convert  the  Analyst’s  War  to  the  Model 

6.13.1  This  is  very  important.  Hartley 

6.14  Must  Incorporate  IV&V  Concurrent  with  Design  and  Test 

6.14.1  This  is  also  very  important.  Hartley 

7.  MAJOR  ISSUES 

7.1  Input  Tools 

7.1.1  The  MNS  goal  of  rapid  turnaround  generates  a  requirement  of  either  an 

impractically  small  input  data  stream  or  a  sophisticated  set  of  user  tools  for 
creating,  modifying  and  validating  the  FTLM  input.  This  facility  should  be 
planned  now  in  the  same  way  the  model  is  planned.  Hartley 

7.1.2  Determine  causes  for  long  setup  times  (e.g.,  technology,  personnel,  data 

collection,  or  other).  Consider  some  means  to  duplicate  or  replicate  scenarios 
to  help  reduce  or  minimize  the  setup  time.  Consider  use  of  parallel  or 
distributed  processing  for  timeliness.  Develop  some  means  to  decompose  the 
model  to  help  minimize  the  complexity  and  to  support  parallel  and/or  distributed 
processing.  Seems  that  an  application-oriented  GUI  would  be  nice. 
Computer-aided  training  module  that  incorporates  the  use  of  optical  disk 
technologies  to  support  training  and  on-line  help.  NOTE:  The  computing 
requirements  defined  in  the  MNS,  and  in  the  PMP,  may  prohibit  use  of  some  of 
the  above  computing  technologies.  Thomas 

7.1.3  In  addition  to  describing  the  model  workings,  an  equal  if  not  greater  amount  of 

effort  should  be  spent  describing  how  the  user  interface  will  allow  the  J-8 
members  to  meaningfully  run  the  model  themselves  -  without  a  supporting  cast 
of  thousands.  That  will  require  new  levels  of  sophistication  in  the  man-machine 
interface.  Martellaro 

7.1.4  When  considering  a  stochastic  model  and  variable  outputs,  the  difficulties  of 

variable  inputs  are  sometimes  overlooked.  While  the  output  result  can  be  a 
simple  number  (for  example,  70%  chance  of  success),  the  user  interface  required 
to  propagate  alternative  inputs  into  the  code  is  ignored  -  and  that’s  why  previous 
models  take  so  long  to  set  up.  Martellaro 

7.1.5  Need  a  design  for  the  user  interface.  It  needs  to  be  done  RIGHT  AWAY.  A  lot 

of  the  programming  will  be  dictated  by  how  the  user  is  going  to  interact  with  the 
system,  and  trying  to  add  that  later  will  be  a  fatal  mistake.  This  is  the  most 
important  thing  I  can  say  and  the  most  likely  cause  for  failure  of  the  project 
later.  Martellaro 

7.1.6  Lots  of  comments  about  user  interface,  input  to  make  easy  and  easy  to  validate 

and  output  to  make  analysis  easy,  consistent,  complete,  etc.,  especially 
considering  comparison  of  variations.  Hartley 

7.1.7  Set  up  forces  -  this  really  requires  a  GUI  and  extensive  source  data.  The  GUI 
would  lead  the  analyst  rapidly  through  the  set  up  steps,  offering  menus  and  radio 
buttons  to  select  from  predictable  options  and  collecting  non-standard  elements 
to  offer  during  future  excursions.  A  common  US  DOD  and  or  Jane’s  Library 
data  base  should  be  the  root  source  with  tailored  and  tailorable  additions.  A 
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candidate  source  is  the  combination  of  Conventional  Force  Data  Base  (CFDB) 
and  Master  Simulation  Data  System  (MSDS).  Once  generic  units  are 
available,  placing  them  is  a  relatively  easy  activity  when  supported  by  a 
map-based  GUI.  Determining  and  entering  the  unit  orientation  (direction  of 
majority  of  combat  effectiveness)  and  support  relationship  is  more  difficult. 
Particularly  in  a  joint  or  combined  scenario,  the  following  type  of  questions  are 
hard  to  answer,  but  terribly  important:  Which  support  elements  are  responsible 
for  which  combat  elements?  Which  are  interchangeable?  Which  can  support 
multiple  combat  units?  For  how  long?  This  type  of  detail  is  often  required  and 
available  only  from  the  parent  command  of  the  units  involved.  A  relational 
and/or  object-oriented  data  base  is  recommended.  Units  of  a  given  type  should 
have  given  characteristics  and  performance,  but  unique  units  should  be 
" creatable "  by  inheriting  the  generic  characteristics  and  adding  special  elements 
and  functions.  Packard 

7.1.8  One  of  the  major  things  to  be  learned  from  CASTFOREM  is  the  need  for 
designing  a  good  user  interface  at  the  front  end.  This  greatly  reduces  the  total 
system  cost,  increases  the  flexibility  of  the  model,  vastly  decreases  the  turn 
around  time  for  a  study,  permits  more  and  better  post  run  analyses,  and  ensures 
better  quality  by  reducing  errors  in  the  study.  Hartley 

7.2  Data  Storage 

7.2.1  How  may  people  and  months  of  preparation  do  they  expect  FTLM  to  require  to 

run.  I  see  no  discussions  on  the  possibility  of  using  a  database  for  the 
parameters  and  probabilities.  And  what  consideration  have  they  given  to  an 
input  and  editing  tools  for  scenarios  and  such.  Kruse 

7.2.2  FTLM  will  need  output  support  system  to  capture  output  data  and  support 

rapid,  reliable  and  meaningful  analysis  of  the  complex  output.  Hartley 

7.3  Analytic  Tools 

7.3.1  Similarly,  the  output  analysis  needs  to  be  planned  now.  Hartley 

7.3.2  Page  7:  FTLM  output  should  be  designed  with  statistical  software 

interoperability  in  mind,  be  it  SAS  or  another.  Packard 

7.3.3  Territorial  ownership  as  an  MOE  could  be  calculated  using  a  concept  of  cellular 

automata:  neighbors.  That  is,  node  and  link  ownership  depends  on  proximity 
and/or  time  since  last  occupancy  by  a  combatant.  Hartley 

7.3.4  Must  think  a  lot  about  sensitivity  analyses.  What  model  interactions  drive  the 

variability?  There  should  be  a  sub-process  built  into  the  model  which  can 
estimate  the  influence  of  any  variable.  Then  it  can  be  turned  off  and  the  model 
will  run  faster  with  no  decrease  in  fidelity.  Martellaro 

7.3.5  He  gets  it  exactly  right:  " How  sensitive  are  my  conclusions  to  the  values  I 

selected ."  Martellaro 


7.4 


Environment  Definition 
7.4.1  Needs  content. 


Hartley 
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7.5  Decision  Logic 

7.5.1  There  is  not  enough  discussion  about  the  rule  sets  and  how  they  are  to  be 

implemented.  How  are  the  rule  sets  specified.  Are  they  specified  as  in  expert 
system  rule  base  type  rules  such  as  "if  this  happens  then  do  that ”  or  are  any  of 
the  rules  based  in  a  more  mathematical  type  algorithm  ( example ,  like  in  Glenn 
A  llgood ’s  work )  ?  Kruse 

7.5.2  Constructing  the  rule  sets  for  FILM  will  be  a  huge  undertaking.  How  will  the 

rule  sets  be  validated  (and  by  whom),  especially  for  completeness?  How  will 
these  rule  sets  be  constructed?  Will  an  expert  system  or  neural  net  approach  be 
used?  Since  there  will  be  so  many  rules  to  process,  perhaps  a  pruning 
methodology  should  be  used  to  eliminate  testing  all  the  rules.  Only  rules 
pertaining  to  a  certain  situation  (a  grouping)  would  be  tested:  a  branch  of  the 
decision  tree  would  be  traversed.  Turley 

7.5.3  Have  to  watch  for  " grandfathered"  rule  sets  becoming  used  without 

understanding  vs.  the  problem  of  users  not  wanting  to  use  the  model  because  it 
would  be  ” too  hard”  to  come  up  with  rules.  Hartley 

7.5.4  1  worry  about  the  advisability  of  not  accepting  feedback  from  the  field  in  a  high 

level  COA  device.  The  tendency  is  to  want  to  model  processes  that  represent 
large-scale  actions  in  order  to  force  stochasticism.  For  a  more  detailed  model, 
I  can  do  that.  I  can  model  a  tank  main  gun  and  estimate,  based  on  weapons 
data,  what  the  probability  is  that  a  round  will  strike  an  enemy  tank  in  the  turret, 
turret  ring  or  the  tracks.  1  do  that  because  I  can’t  ask  the  round  where  it’s  going 
to  hit.  (No  humor  intended.)  But  I  *can*  pick  up  the  phone  and  ask  a 
Battalion  Commander  what  he  intends  to  do.  He  says,  ”if  it  rains  before  sunset, 
1  advance.  If  not,  I  wait  until  dark.  And  I’m  doing  that  because....)  You  have 
to  take  things  like  that  into  consideration  when  generating  stochastic  runs  on  a 
Theater  level  model.  Otherwise,  the  model  becomes  hopelessly  disconnected 
from  reality.  Martellaro 

7.5.5  Document  says  "Constraints  representing  part  of  the  air  campaign  plan  (COA) 

may  be  established  by  the  user  to  restrict  the  valid  path  set.”  Are  these 
constraints  given  in  the  form  of  a  generalized  rule  set  that  the  user  adds  to  the 
model  rule  set,  or  are  they  part  of  a  particular  mission  plan?  Kruse 

Availability 

Will  the  data  needed  for  FTLM  be  obtainable?  Have  the  data  been  identified 
(in  any  way)?  Hartley 

Pages  2-3  address  US  forces.  The  database  needs  to  be  developed  with  a  new 
world  order  /  combined  force  mentality.  The  Republic  of  Korea  has  55  divisions 
as  compared  with  one  US  division.  Allied  may  have  old  soviet  weapons  and 
training  and  /  or  old  US  weapons.  Opposing  forces  may  have  US  weapons  and 
training.  Neutrals  could  have  anything.  Terrorists  and  less  sophisticated 
combatants  may  used  mixtures  of  weapons,  doctrine,  and  organization 
previously  not  encountered.  Packard 

7.7  Documentation 

7.7. 1  Subsequent  design  documents  should  be  more  specific  in  distinguishing  between 
general  discussion  and  concepts  which  can  be  modeled.  To  identify  weaknesses 
of  older  models  and  combine  that  with  "here’s  what  we  should  do”  arguments 


7.6  Data 

7.6.1 

7.6.2 
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is  not  equal  to  defining  a  methodology  or  basis  for  quantitative  modeling. 

Martellaro 

7.7.2  The  analyst’s  guide  for  the  FTLM  will  need  to  include  guides  for  modeling  the 
scenario  through  choice  of  the  node/link  architecture.  Terrain  with  clearly 
defined  avenues  of  approach  and  objectives  are  relatively  easy  to  model.  There 
are  some  potential  areas  of  difficulty.  For  instance  high  points  may  be  good 
defensive  positions.  In  some  situations  these  will  be  objectives;  however,  they 
may  not  be  actually  nodes  on  the  travel  paths.  It  may  be  good  practice  to  allow 
bypassing  some  objectives  with  a  node  before  and  after  that  node  and  a  link 
around  it.  Alternatively,  a  city  may  be  a  realistic  node  that  may  need  to  be 
taken  or  bypassed. 

A  situation  that  is  difficult  to  model  is  open  territory.  Some  nodes  are  obvious, 
such  as  cities,  fortifications,  etc.  However,  the  paths  that  may  be  taken  are 
infinite.  One  modeling  approach  is  to  generate  many  alternate  links  between 
nodes.  This  has  the  problem  of  opposing  forces  on  parallel  paths  always  being 
unable  to  attack  each  other.  Another  approach  is  to  generate  one  link  with  great 
width  and  an  associated  probability  of  not  seeing  another  force  on  the  link. 
Thus  opposing  forces  might  see  each  other  or  might  miss  each  other.  This  has 
the  problem  of  no  permitting  changes  in  destination  in  mid  trip  (over  perhaps 
long  distances),  which  would  actually  be  possible.  A  solution  could  be  to 
generate  notional  (not  attached  to  any  real  terrain  feature)  fixed  nodes  in  the 
middle  of  such  links,  connected  to  other  mid-link  nodes.  Hartley 

7.7.3  The  analysis  of  the  COAs  in  Schmidt’s  thesis  says  that  CO  A  #3  was  perceived 

as  the  one  being  pursued  even  when  CO  A  #1  and  when  C0A#2  were  the 
actual  COAs  being  pursued.  An  examination  of  the  scenario  shows  that  at  the 
first  part  of  the  action,  Avenues  of  Approach  #1  and  #2  were  confounded  and 
at  the  end  AA  #2  and  #J  were  confounded.  Further,  the  COA  defmirions 
made  distinguishing  very  difficult  because  the  differences  lay  largely  in  the 
actions  of  the  infantry  brigades  and  the  mechanized  brigades,  which  are  more 
difficult  to  detect  and  distinguish.  This  might  be  a  true  representation  of  reality, 
not  a  flaw  in  the  model!  The  FTLM  analyst  documentation  must  make  it  clear 
how  important  it  is  for  the  analyst  to  make  sure  he  or  she  understands  the 
native  distinguishability  of  the  COAs.  Further,  less  distinctive  COAs  might 
require  less  subtlety  in  possible  deception  plans,  otherwise  the  enemy  might 
acquire  the  deception,  believe  it,  yet  still  act  in  what  ends  up  being  the  proper 
fashion  for  the  enemy.  Hartley 

7.8  Management 

7.8.1  Need  a  concept  for  the  development  system.  POSIX  and  X  are  fine.  There 
should  be  a  survey  of  CASE  tools  and  exploration  done  on  how  the  software 
project  and  code  will  be  managed.  The  last  thing  that  should  be  done,  on  a 
model  of  this  importance,  is  to  just  sit  down  and  start  writing  (undisciplined) 
code  in  ”vi”  and  starting  to  compile.  There  needs  to  be  a  Source  Code  Control 
System  agreed  on.  A  system  needs  to  be  developed  for  the  cooperation  but 
non-interference  of  multiple  authors.  For  example,  it  is  critical  that  each 
programmer  be  able  to  exercise  and  test  his/her  own  changes,  but  it  must  be 
done  in  a  common  system  in  which  the  baseline  and  core  code  can  be 
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standardized,  and  controlled.  A  graphical  CASE  tool  which  can  show  the 
relationships  of  the  modules  <4  objects  should  be  explored.  Martellaro 
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